[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:41:42 dist-test-slave-vq2q krb5kdc[31560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:41:42 dist-test-slave-vq2q krb5kdc[31560](info): set up 2 sockets
Aug 07 19:41:42 dist-test-slave-vq2q krb5kdc[31560](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 19:41:43.162770 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:41:43 dist-test-slave-vq2q krb5kdc[31560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:41:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:41:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:41:43.345057 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:34267
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:34267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:43.669211 31574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:43.669509 31574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:43.669649 31574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:43.676820 31574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:41:43.676949 31574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:43.677028 31574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:43.677107 31574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:41:43.677168 31574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:41:43.687057 31574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:34267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:34267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:43.688416 31574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:43.689766 31574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:43.701836 31579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:43.701948 31580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:43.701867 31583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:43.702292 31574 server_base.cc:1034] running on GCE node
I20260807 19:41:43.703140 31574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:43.704574 31574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:43.705780 31574 hybrid_clock.cc:648] HybridClock initialized: now 1786131703705746 us; error 88 us; skew 500 ppm
I20260807 19:41:43.706171 31574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:43 dist-test-slave-vq2q krb5kdc[31560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131703, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:43.709488 31574 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:41:43.711057 31574 webserver.cc:460] Webserver started at http://127.30.208.62:37983/ using document root <none> and password file <none>
I20260807 19:41:43.711733 31574 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:43.711833 31574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:43.712101 31574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:43.715960 31574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "3cfe084f7d774ce391dd6b0737a77b66"
format_stamp: "Formatted at 2026-08-07 19:41:43 on dist-test-slave-vq2q"
server_key: "8ff8f074f3b725baf47ecd56bcf28e16"
server_key_iv: "43bdbfdcc855b6143c7e43b186295965"
server_key_version: "encryptionkey@0"
I20260807 19:41:43.716671 31574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "3cfe084f7d774ce391dd6b0737a77b66"
format_stamp: "Formatted at 2026-08-07 19:41:43 on dist-test-slave-vq2q"
server_key: "8ff8f074f3b725baf47ecd56bcf28e16"
server_key_iv: "43bdbfdcc855b6143c7e43b186295965"
server_key_version: "encryptionkey@0"
I20260807 19:41:43.721961 31574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:41:43.725711 31590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:43.727214 31574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 19:41:43.727386 31574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "3cfe084f7d774ce391dd6b0737a77b66"
format_stamp: "Formatted at 2026-08-07 19:41:43 on dist-test-slave-vq2q"
server_key: "8ff8f074f3b725baf47ecd56bcf28e16"
server_key_iv: "43bdbfdcc855b6143c7e43b186295965"
server_key_version: "encryptionkey@0"
I20260807 19:41:43.727581 31574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:43.752835 31574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:43.757280 31574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:43.757562 31574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:43.778326 31574 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:34267
I20260807 19:41:43.778337 31645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:34267 every 8 connection(s)
I20260807 19:41:43.780026 31574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:41:43.783659 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 31574
I20260807 19:41:43.784013 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:41:43.784893 31646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:43.786593 31552 external_mini_cluster.cc:1428] Setting key a5d2da5ed99d0f90de54e77c96d8a43c
I20260807 19:41:43.797030 31646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66: Bootstrap starting.
I20260807 19:41:43.799993 31646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:43.801229 31646 log.cc:824] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:43.804665 31646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66: No bootstrap required, opened a new log
I20260807 19:41:43.813371 31646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cfe084f7d774ce391dd6b0737a77b66" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34267 } }
I20260807 19:41:43.813725 31646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:43.813827 31646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cfe084f7d774ce391dd6b0737a77b66, State: Initialized, Role: FOLLOWER
I20260807 19:41:43.814406 31646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cfe084f7d774ce391dd6b0737a77b66" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34267 } }
I20260807 19:41:43.814610 31646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:41:43.814736 31646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:41:43.814882 31646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 19:41:43 dist-test-slave-vq2q krb5kdc[31560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:43.817099 31646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cfe084f7d774ce391dd6b0737a77b66" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34267 } }
I20260807 19:41:43.817801 31646 leader_election.cc:302] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cfe084f7d774ce391dd6b0737a77b66; no voters:
I20260807 19:41:43.818449 31646 leader_election.cc:288] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:41:43.818580 31652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:41:43.819523 31652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [term 1 LEADER]: Becoming Leader. State: Replica: 3cfe084f7d774ce391dd6b0737a77b66, State: Running, Role: LEADER
I20260807 19:41:43.820109 31652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cfe084f7d774ce391dd6b0737a77b66" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34267 } }
I20260807 19:41:43.820930 31646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:41:43.823148 31653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cfe084f7d774ce391dd6b0737a77b66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cfe084f7d774ce391dd6b0737a77b66" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34267 } } }
I20260807 19:41:43.823786 31653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [sys.catalog]: This master's current role is: LEADER
I20260807 19:41:43.823417 31654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cfe084f7d774ce391dd6b0737a77b66. Latest consensus state: current_term: 1 leader_uuid: "3cfe084f7d774ce391dd6b0737a77b66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cfe084f7d774ce391dd6b0737a77b66" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34267 } } }
I20260807 19:41:43.829424 31654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66 [sys.catalog]: This master's current role is: LEADER
I20260807 19:41:43.830880 31649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:43.791475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49718 (local address 127.30.208.62:34267)
0807 19:41:43.792396 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:43.792411 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:43.793713 (+ 1302us) server_negotiation.cc:407] Connection header received
0807 19:41:43.802893 (+ 9180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:43.802919 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:43.803382 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:43.803818 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:43.806579 (+ 2761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:43.808671 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:43.810400 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:43.811451 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:43.811627 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:43.817256 (+ 5629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:43.817270 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:43.817289 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:43.817337 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:43.821685 (+ 4348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:43.822932 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:43.822942 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:43.822951 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:43.823547 (+ 596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:43.824257 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:43.824263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:43.824267 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:43.825226 (+ 959us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:43.825745 (+ 519us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:43.828807 (+ 3062us) server_negotiation.cc:299] Negotiation successful
0807 19:41:43.829313 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":408,"threads_started":1}
I20260807 19:41:43.834769 31657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:41:43.841131 31657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:41:43.851919 31657 catalog_manager.cc:1347] Generated new cluster ID: 74cd89e7b99245e18ae26c238d36f1a7
I20260807 19:41:43.852043 31657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:41:43.883862 31657 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:41:43.885478 31657 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:41:43.899580 31657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cfe084f7d774ce391dd6b0737a77b66: Generated new TSK 0
I20260807 19:41:43.900472 31657 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:41:43.963109 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34267
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:44.328346 31673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:44.328655 31673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:44.328794 31673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:44.336520 31673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:44.336640 31673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:44.336807 31673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:41:44.346910 31673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34267
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:44.348299 31673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:44.349678 31673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:44.362198 31679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:44.362275 31680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:44.362246 31682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:44.363211 31673 server_base.cc:1034] running on GCE node
I20260807 19:41:44.363804 31673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:44.364773 31673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:44.365971 31673 hybrid_clock.cc:648] HybridClock initialized: now 1786131704365923 us; error 41 us; skew 500 ppm
I20260807 19:41:44.366271 31673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:44 dist-test-slave-vq2q krb5kdc[31560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131704, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:44.369376 31673 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:41:44.370759 31673 webserver.cc:460] Webserver started at http://127.30.208.1:36425/ using document root <none> and password file <none>
I20260807 19:41:44.371392 31673 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:44.371492 31673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:44.371753 31673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:44.374598 31673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "f1c92f99434d4116a408b9c84c54a4e9"
format_stamp: "Formatted at 2026-08-07 19:41:44 on dist-test-slave-vq2q"
server_key: "a04f3de80cde12bfffba0319f54a46e6"
server_key_iv: "d45b971ca1c3fc12f13265e7a1bd62c1"
server_key_version: "encryptionkey@0"
I20260807 19:41:44.375308 31673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "f1c92f99434d4116a408b9c84c54a4e9"
format_stamp: "Formatted at 2026-08-07 19:41:44 on dist-test-slave-vq2q"
server_key: "a04f3de80cde12bfffba0319f54a46e6"
server_key_iv: "d45b971ca1c3fc12f13265e7a1bd62c1"
server_key_version: "encryptionkey@0"
I20260807 19:41:44.380522 31673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 19:41:44.384038 31689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:44.385416 31673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:41:44.385552 31673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "f1c92f99434d4116a408b9c84c54a4e9"
format_stamp: "Formatted at 2026-08-07 19:41:44 on dist-test-slave-vq2q"
server_key: "a04f3de80cde12bfffba0319f54a46e6"
server_key_iv: "d45b971ca1c3fc12f13265e7a1bd62c1"
server_key_version: "encryptionkey@0"
I20260807 19:41:44.385739 31673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:44.420418 31673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:44.424754 31673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:44.425014 31673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:44.426035 31673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:41:44.427380 31673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:44.427467 31673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:44.427572 31673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:44.427614 31673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:44.458220 31673 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:45163
I20260807 19:41:44.458287 31811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:45163 every 8 connection(s)
I20260807 19:41:44.459487 31673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:41:44.465099 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 31673
I20260807 19:41:44.465329 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:41:44.465942 31552 external_mini_cluster.cc:1428] Setting key 8a6517c226f43895d5902933df606ccc
Aug 07 19:41:44 dist-test-slave-vq2q krb5kdc[31560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131704, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:44.479528 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:44.462023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:50783 (local address 127.30.208.62:34267)
0807 19:41:44.462514 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:44.462520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:44.463390 (+ 870us) server_negotiation.cc:407] Connection header received
0807 19:41:44.465166 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:44.465172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:44.465285 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:44.465380 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:44.466999 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:44.467881 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:44.469440 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:44.469982 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:44.470115 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:44.473314 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:44.473320 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:44.473325 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:44.473364 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:44.476236 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:44.477080 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:44.477084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:44.477086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:44.477247 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:44.477727 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:44.477731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:44.477734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:44.478065 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:44.478203 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:44.479065 (+ 862us) server_negotiation.cc:299] Negotiation successful
0807 19:41:44.479247 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":221,"threads_started":1}
I20260807 19:41:44.481056 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:44.462524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34267 (local address 127.30.208.1:50783)
0807 19:41:44.463217 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:44.463266 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:44.464904 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:44.465637 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:44.465645 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:44.466197 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:44.466843 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:44.466864 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:44.468048 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:44.468051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:44.469307 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:44.469319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:44.470169 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:44.470173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:44.470380 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:44.470989 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:44.471016 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:44.473074 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:44.476379 (+ 3305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:44.476388 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:44.476405 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:44.476965 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:44.477346 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:44.477351 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:44.477354 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:44.477590 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:44.478230 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:44.478238 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:44.478656 (+ 418us) client_negotiation.cc:769] Sending connection context
0807 19:41:44.478963 (+ 307us) client_negotiation.cc:240] Negotiation successful
0807 19:41:44.479512 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":267,"threads_started":1}
I20260807 19:41:44.482676 31812 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34267
I20260807 19:41:44.483016 31812 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:44.483937 31812 heartbeater.cc:511] Master 127.30.208.62:34267 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
I20260807 19:41:44.486256 31608 ts_manager.cc:194] Registered new tserver with Master: f1c92f99434d4116a408b9c84c54a4e9 (127.30.208.1:45163)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
I20260807 19:41:44.487920 31608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:50783
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:41:44.510577 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34267
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:44.836645 31818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:44.836949 31818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:44.837049 31818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:44.844367 31818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:44.844485 31818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:44.844602 31818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:41:44.854528 31818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34267
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:44.855801 31818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:44.857102 31818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:44.870087 31824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:44.870162 31825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:44.870113 31827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:44.871146 31818 server_base.cc:1034] running on GCE node
I20260807 19:41:44.871729 31818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:44.872772 31818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:44.873983 31818 hybrid_clock.cc:648] HybridClock initialized: now 1786131704873924 us; error 43 us; skew 500 ppm
I20260807 19:41:44.874289 31818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:44 dist-test-slave-vq2q krb5kdc[31560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131704, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:44.877430 31818 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:41:44.878958 31818 webserver.cc:460] Webserver started at http://127.30.208.2:36711/ using document root <none> and password file <none>
I20260807 19:41:44.879623 31818 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:44.879725 31818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:44.879999 31818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:44.883023 31818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "04205fdd671e43dfa3b5514aaaf6fac0"
format_stamp: "Formatted at 2026-08-07 19:41:44 on dist-test-slave-vq2q"
server_key: "796461e043466f145deac5d3fb3f54ac"
server_key_iv: "802fd4c67fdb11b7ecc0c07595e7de6f"
server_key_version: "encryptionkey@0"
I20260807 19:41:44.883811 31818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "04205fdd671e43dfa3b5514aaaf6fac0"
format_stamp: "Formatted at 2026-08-07 19:41:44 on dist-test-slave-vq2q"
server_key: "796461e043466f145deac5d3fb3f54ac"
server_key_iv: "802fd4c67fdb11b7ecc0c07595e7de6f"
server_key_version: "encryptionkey@0"
I20260807 19:41:44.889036 31818 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 19:41:44.892603 31834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:44.894032 31818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:41:44.894194 31818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "04205fdd671e43dfa3b5514aaaf6fac0"
format_stamp: "Formatted at 2026-08-07 19:41:44 on dist-test-slave-vq2q"
server_key: "796461e043466f145deac5d3fb3f54ac"
server_key_iv: "802fd4c67fdb11b7ecc0c07595e7de6f"
server_key_version: "encryptionkey@0"
I20260807 19:41:44.894400 31818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:44.907886 31818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:44.912004 31818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:44.912299 31818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:44.913345 31818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:41:44.914740 31818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:44.914824 31818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:44.914948 31818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:44.915004 31818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:44.947078 31818 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:36793
I20260807 19:41:44.947129 31954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:36793 every 8 connection(s)
I20260807 19:41:44.948338 31818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:41:44.958618 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 31818
I20260807 19:41:44.958799 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:41:44.959241 31552 external_mini_cluster.cc:1428] Setting key 534e4bca696c453e77c0eff9d1157e86
Aug 07 19:41:44 dist-test-slave-vq2q krb5kdc[31560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131704, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:44.970674 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:44.950824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:41071 (local address 127.30.208.62:34267)
0807 19:41:44.951037 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:44.951043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:44.952266 (+ 1223us) server_negotiation.cc:407] Connection header received
0807 19:41:44.954124 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:44.954129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:44.954201 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:44.954289 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:44.955833 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:44.956703 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:44.959447 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:44.959896 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:44.960017 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:44.964681 (+ 4664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:44.964687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:44.964690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:44.964727 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:44.967304 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:44.968110 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:44.968116 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:44.968119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:44.968287 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:44.968793 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:44.968798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:44.968802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:44.969147 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:44.969335 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:44.970101 (+ 766us) server_negotiation.cc:299] Negotiation successful
0807 19:41:44.970346 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:41:44.971529 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:44.951358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34267 (local address 127.30.208.2:41071)
0807 19:41:44.952093 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:44.952132 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:44.953863 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:44.954509 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:44.954517 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:44.955061 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:44.955657 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:44.955674 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:44.957044 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:44.957049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:44.959268 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:44.959294 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:44.960138 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:44.960143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:44.960442 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:44.961538 (+ 1096us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:44.961569 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:44.964426 (+ 2857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:44.967486 (+ 3060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:44.967498 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:44.967513 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:44.967973 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:44.968456 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:44.968460 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:44.968463 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:44.968676 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:44.969316 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:44.969322 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:44.969675 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 19:41:44.969950 (+ 275us) client_negotiation.cc:240] Negotiation successful
0807 19:41:44.970375 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":280,"threads_started":1}
I20260807 19:41:44.973114 31955 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34267
I20260807 19:41:44.973505 31955 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:44.974421 31955 heartbeater.cc:511] Master 127.30.208.62:34267 requested a full tablet report, sending...
I20260807 19:41:44.976071 31608 ts_manager.cc:194] Registered new tserver with Master: 04205fdd671e43dfa3b5514aaaf6fac0 (127.30.208.2:36793)
I20260807 19:41:44.976907 31608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:41071
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:41:45.004909 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34267
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:45.333851 31960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:45.334163 31960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:45.334311 31960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:45.341619 31960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:45.341763 31960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:45.341928 31960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:41:45.351888 31960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34267
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:45.353224 31960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:45.354547 31960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:45.367328 31966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:45.367400 31967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:45.367375 31969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:45.368244 31960 server_base.cc:1034] running on GCE node
I20260807 19:41:45.368902 31960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:45.369910 31960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:45.371124 31960 hybrid_clock.cc:648] HybridClock initialized: now 1786131705371068 us; error 45 us; skew 500 ppm
I20260807 19:41:45.371446 31960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:45 dist-test-slave-vq2q krb5kdc[31560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131705, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:45.374627 31960 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:41:45.376108 31960 webserver.cc:460] Webserver started at http://127.30.208.3:33673/ using document root <none> and password file <none>
I20260807 19:41:45.376750 31960 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:45.376863 31960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:45.377205 31960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:45.380302 31960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "c3257003589043a7b7e1e7108beab07e"
format_stamp: "Formatted at 2026-08-07 19:41:45 on dist-test-slave-vq2q"
server_key: "19d67eab3cd19a93bbd88c7de0fd7d88"
server_key_iv: "d8cf4df2712c1967ea87ae5c3c334c1a"
server_key_version: "encryptionkey@0"
I20260807 19:41:45.381067 31960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "c3257003589043a7b7e1e7108beab07e"
format_stamp: "Formatted at 2026-08-07 19:41:45 on dist-test-slave-vq2q"
server_key: "19d67eab3cd19a93bbd88c7de0fd7d88"
server_key_iv: "d8cf4df2712c1967ea87ae5c3c334c1a"
server_key_version: "encryptionkey@0"
I20260807 19:41:45.386513 31960 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:41:45.390111 31977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:45.391533 31960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:41:45.391693 31960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "c3257003589043a7b7e1e7108beab07e"
format_stamp: "Formatted at 2026-08-07 19:41:45 on dist-test-slave-vq2q"
server_key: "19d67eab3cd19a93bbd88c7de0fd7d88"
server_key_iv: "d8cf4df2712c1967ea87ae5c3c334c1a"
server_key_version: "encryptionkey@0"
I20260807 19:41:45.391886 31960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:45.432360 31960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:45.436890 31960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:45.437203 31960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:45.438210 31960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:41:45.439556 31960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:45.439637 31960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:45.439774 31960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:45.439831 31960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:45.470944 31960 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:37931
I20260807 19:41:45.470999 32095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:37931 every 8 connection(s)
I20260807 19:41:45.472223 31960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:41:45.473899 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 31960
I20260807 19:41:45.474094 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:41:45.474589 31552 external_mini_cluster.cc:1428] Setting key 33fc548116fbb0b991f2a657cad757a2
Aug 07 19:41:45 dist-test-slave-vq2q krb5kdc[31560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131705, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:45.491271 31812 heartbeater.cc:503] Master 127.30.208.62:34267 was elected leader, sending a full tablet report...
I20260807 19:41:45.493214 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:45.474887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:60799 (local address 127.30.208.62:34267)
0807 19:41:45.475367 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:45.475373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:45.476850 (+ 1477us) server_negotiation.cc:407] Connection header received
0807 19:41:45.478708 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:45.478714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:45.478798 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:45.478893 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:45.480527 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:45.481362 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:45.482901 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:45.483316 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:45.483434 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:45.486881 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:45.486886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:45.486889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:45.486923 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:45.489515 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:45.490441 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:45.490445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:45.490448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:45.490608 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:45.491518 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:45.491524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:45.491527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:45.491872 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:45.492033 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:45.492748 (+ 715us) server_negotiation.cc:299] Negotiation successful
0807 19:41:45.492923 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":218,"threads_started":1}
I20260807 19:41:45.494266 32100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:45.475383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34267 (local address 127.30.208.3:60799)
0807 19:41:45.476208 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:45.476256 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:45.478454 (+ 2198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:45.479137 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:45.479145 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:45.479678 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:45.480366 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:45.480386 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:45.481509 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:45.481515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:45.482751 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:45.482765 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:45.483539 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:45.483547 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:45.483762 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:45.484379 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:45.484401 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:45.486666 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:45.489679 (+ 3013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:45.489689 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:45.489708 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:45.490332 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:45.490711 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:45.490714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:45.490717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:45.491002 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:45.492017 (+ 1015us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:45.492024 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:45.492346 (+ 322us) client_negotiation.cc:769] Sending connection context
0807 19:41:45.492641 (+ 295us) client_negotiation.cc:240] Negotiation successful
0807 19:41:45.493142 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":307,"threads_started":1}
I20260807 19:41:45.495522 32097 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34267
I20260807 19:41:45.495836 32097 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:45.496559 32097 heartbeater.cc:511] Master 127.30.208.62:34267 requested a full tablet report, sending...
I20260807 19:41:45.497867 31608 ts_manager.cc:194] Registered new tserver with Master: c3257003589043a7b7e1e7108beab07e (127.30.208.3:37931)
I20260807 19:41:45.498677 31608 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:60799
I20260807 19:41:45.503150 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:41:45 dist-test-slave-vq2q krb5kdc[31560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131705, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:41:45 dist-test-slave-vq2q krb5kdc[31560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131705, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:41:45.551990 32106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:45.535619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38812 (local address 127.30.208.1:45163)
0807 19:41:45.536955 (+ 1336us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:45.536962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:45.536989 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:41:45.537053 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:45.537059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:45.537336 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:45.537480 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:45.539076 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:45.540198 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:45.541607 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:45.542217 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:45.542336 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:45.546471 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:45.546485 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:45.546500 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:45.546537 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:45.549401 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:45.550086 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:45.550093 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:45.550095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:45.550248 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:45.550633 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:45.550636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:45.550638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:45.551002 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:45.551252 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:45.551476 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 19:41:45.551685 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1181,"thread_start_us":1020,"threads_started":1}
W20260807 19:41:45.553386 31720 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38812
Aug 07 19:41:45 dist-test-slave-vq2q krb5kdc[31560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:41:45 dist-test-slave-vq2q krb5kdc[31560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:41:45.603775 32106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:45.586400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38828 (local address 127.30.208.1:45163)
0807 19:41:45.586615 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:45.586622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:45.586901 (+ 279us) server_negotiation.cc:407] Connection header received
0807 19:41:45.587078 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:45.587082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:45.587164 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:45.587262 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:45.588673 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:45.589456 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:45.590745 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:45.591166 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:45.591282 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:45.598850 (+ 7568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:45.598855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:45.598858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:45.598899 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:45.601345 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:45.601961 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:45.601968 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:45.601970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:45.602097 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:45.602452 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:45.602456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:45.602458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:45.602722 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:45.602945 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:45.603243 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 19:41:45.603468 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:41:45.605243 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 31673
I20260807 19:41:45.617619 32106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:45.606272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38840 (local address 127.30.208.1:45163)
0807 19:41:45.606425 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:45.606429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:45.606634 (+ 205us) server_negotiation.cc:407] Connection header received
0807 19:41:45.606764 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:45.606768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:45.606824 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:45.606897 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:45.608001 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:45.608671 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:45.609941 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:45.610357 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:45.610448 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:45.612595 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:45.612600 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:45.612603 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:45.612635 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:45.615310 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:45.615963 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:45.615968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:45.615970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:45.616126 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:45.616460 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:45.616463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:45.616464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:45.616713 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:45.616902 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:45.617098 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 19:41:45.617315 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:41:45.618321 31807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:45.702062 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 31673
I20260807 19:41:45.718784 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 31818
Aug 07 19:41:45 dist-test-slave-vq2q krb5kdc[31560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:41:45.736004 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:45.719805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56844 (local address 127.30.208.2:36793)
0807 19:41:45.720316 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:45.720323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:45.720350 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:41:45.720408 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:45.720414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:45.720618 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:45.720739 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:45.722093 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:45.723207 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:45.724514 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:45.725140 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:45.725396 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:45.729665 (+ 4269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:45.729678 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:45.729694 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:45.729734 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:45.732900 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:45.733764 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:45.733778 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:45.733780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:45.733925 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:45.734395 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:45.734400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:45.734402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:45.734823 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:45.735076 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:45.735409 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 19:41:45.735630 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":235,"threads_started":1}
I20260807 19:41:45.737402 31948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:45.824507 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 31818
I20260807 19:41:45.842027 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 31960
Aug 07 19:41:45 dist-test-slave-vq2q krb5kdc[31560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:41:45.858431 32116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:45.843108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.30.208.3:37931)
0807 19:41:45.843843 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:45.843851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:45.843882 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:41:45.843944 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:45.843950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:45.844155 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:45.844299 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:45.845657 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:45.846766 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:45.848044 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:45.848730 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:45.848845 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:45.852994 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:45.853008 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:45.853023 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:45.853062 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:45.855887 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:45.856546 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:45.856554 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:45.856556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:45.856684 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:45.856963 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:45.856967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:45.856969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:45.857381 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:45.857626 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:45.857846 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 19:41:45.858065 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":399,"threads_started":1}
I20260807 19:41:45.859680 32090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:45.944007 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 31960
I20260807 19:41:45.960579 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 31574
I20260807 19:41:45.961810 31641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:46.067016 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 31574
2026-08-07T19:41:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3989 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:41:46 dist-test-slave-vq2q krb5kdc[32121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:41:46 dist-test-slave-vq2q krb5kdc[32121](info): set up 2 sockets
Aug 07 19:41:46 dist-test-slave-vq2q krb5kdc[32121](info): commencing operation
krb5kdc: starting...
W20260807 19:41:47.175997 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:41:47 dist-test-slave-vq2q krb5kdc[32121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:41:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:41:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:41:47.369812 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:45951
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:38611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:45951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:47.696837 32137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:47.697167 32137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:47.697327 32137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:47.704741 32137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:41:47.704881 32137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:47.704967 32137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:47.705047 32137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:41:47.705108 32137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:41:47.715291 32137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:38611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:45951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:45951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:47.716643 32137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:47.718035 32137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:47.729422 32143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:47.730044 32145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:47.730204 32142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:47.730608 32137 server_base.cc:1034] running on GCE node
I20260807 19:41:47.731426 32137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:47.732882 32137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:47.734193 32137 hybrid_clock.cc:648] HybridClock initialized: now 1786131707734128 us; error 55 us; skew 500 ppm
I20260807 19:41:47.734582 32137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:47 dist-test-slave-vq2q krb5kdc[32121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131707, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:47.737802 32137 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:41:47.739411 32137 webserver.cc:460] Webserver started at http://127.30.208.62:45845/ using document root <none> and password file <none>
I20260807 19:41:47.740156 32137 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:47.740283 32137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:47.740594 32137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:47.743615 32137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "d17382da6c194807901bb9fea8e09fa1"
format_stamp: "Formatted at 2026-08-07 19:41:47 on dist-test-slave-vq2q"
server_key: "83b6036053d723a8fd6083346b76970a"
server_key_iv: "2df3e3b7bce30978281fd3e2a1397e1b"
server_key_version: "encryptionkey@0"
I20260807 19:41:47.744365 32137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "d17382da6c194807901bb9fea8e09fa1"
format_stamp: "Formatted at 2026-08-07 19:41:47 on dist-test-slave-vq2q"
server_key: "83b6036053d723a8fd6083346b76970a"
server_key_iv: "2df3e3b7bce30978281fd3e2a1397e1b"
server_key_version: "encryptionkey@0"
I20260807 19:41:47.749536 32137 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 19:41:47.752969 32154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:47.754400 32137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:41:47.754568 32137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "d17382da6c194807901bb9fea8e09fa1"
format_stamp: "Formatted at 2026-08-07 19:41:47 on dist-test-slave-vq2q"
server_key: "83b6036053d723a8fd6083346b76970a"
server_key_iv: "2df3e3b7bce30978281fd3e2a1397e1b"
server_key_version: "encryptionkey@0"
I20260807 19:41:47.754755 32137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:47.776453 32137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:47.782092 32137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:47.782339 32137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:47.799640 32137 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:45951
I20260807 19:41:47.799660 32207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:45951 every 8 connection(s)
I20260807 19:41:47.800963 32137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:41:47.804759 32208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:47.808847 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 32137
I20260807 19:41:47.809088 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:41:47.809942 31552 external_mini_cluster.cc:1428] Setting key a99c294a79fd0982d74aa91e415cbd20
I20260807 19:41:47.816954 32208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1: Bootstrap starting.
I20260807 19:41:47.820616 32208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:47.821851 32208 log.cc:824] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:41:47 dist-test-slave-vq2q krb5kdc[32121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:47.825160 32208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1: No bootstrap required, opened a new log
I20260807 19:41:47.832872 32208 raft_consensus.cc:348] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17382da6c194807901bb9fea8e09fa1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45951 } }
I20260807 19:41:47.833253 32208 raft_consensus.cc:374] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:47.833353 32208 raft_consensus.cc:729] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d17382da6c194807901bb9fea8e09fa1, State: Initialized, Role: FOLLOWER
I20260807 19:41:47.833472 32211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:47.812059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.30.208.62:45951)
0807 19:41:47.812846 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:47.812857 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:47.812899 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:41:47.815208 (+ 2309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:47.815234 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:47.815645 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:47.815970 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:47.817837 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:47.819176 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:47.820904 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:47.821693 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:47.821850 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:47.826213 (+ 4363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:47.826221 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:47.826236 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:47.826291 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:47.829397 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:47.830073 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:47.830080 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:47.830086 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:47.830253 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:47.830624 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:47.830631 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:47.830633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:47.831179 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:47.831467 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:47.831980 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 19:41:47.832396 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":371,"threads_started":1}
I20260807 19:41:47.834003 32208 consensus_queue.cc:260] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17382da6c194807901bb9fea8e09fa1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45951 } }
I20260807 19:41:47.834285 32208 raft_consensus.cc:388] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:41:47.834429 32208 raft_consensus.cc:482] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:41:47.834601 32208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:47.837085 32208 raft_consensus.cc:504] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17382da6c194807901bb9fea8e09fa1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45951 } }
I20260807 19:41:47.837579 32208 leader_election.cc:302] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d17382da6c194807901bb9fea8e09fa1; no voters:
I20260807 19:41:47.838085 32208 leader_election.cc:288] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:41:47.838212 32213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:41:47.839588 32213 raft_consensus.cc:686] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [term 1 LEADER]: Becoming Leader. State: Replica: d17382da6c194807901bb9fea8e09fa1, State: Running, Role: LEADER
I20260807 19:41:47.840143 32208 sys_catalog.cc:564] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:41:47.840137 32213 consensus_queue.cc:237] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17382da6c194807901bb9fea8e09fa1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45951 } }
I20260807 19:41:47.842976 32216 sys_catalog.cc:455] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d17382da6c194807901bb9fea8e09fa1. Latest consensus state: current_term: 1 leader_uuid: "d17382da6c194807901bb9fea8e09fa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17382da6c194807901bb9fea8e09fa1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45951 } } }
I20260807 19:41:47.843334 32216 sys_catalog.cc:458] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [sys.catalog]: This master's current role is: LEADER
I20260807 19:41:47.843807 32215 sys_catalog.cc:455] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d17382da6c194807901bb9fea8e09fa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d17382da6c194807901bb9fea8e09fa1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45951 } } }
I20260807 19:41:47.844108 32215 sys_catalog.cc:458] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1 [sys.catalog]: This master's current role is: LEADER
I20260807 19:41:47.845296 32224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:41:47.850337 32224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:41:47.859573 32224 catalog_manager.cc:1347] Generated new cluster ID: 3543c8e25e294c70bfbb93b7a96ee1c7
I20260807 19:41:47.859697 32224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:41:47.870486 32224 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:41:47.872300 32224 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:41:47.883080 32224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d17382da6c194807901bb9fea8e09fa1: Generated new TSK 0
I20260807 19:41:47.884135 32224 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:41:47.930498 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45951
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:38611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:48.264338 32237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:48.264639 32237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:48.264791 32237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:48.272035 32237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:48.272157 32237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:48.272330 32237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:41:48.281997 32237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:38611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45951
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:48.283375 32237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:48.284721 32237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:48.297519 32242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:48.297510 32245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:48.297544 32243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:48.298496 32237 server_base.cc:1034] running on GCE node
I20260807 19:41:48.299154 32237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:48.300078 32237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:48.301287 32237 hybrid_clock.cc:648] HybridClock initialized: now 1786131708301233 us; error 44 us; skew 500 ppm
I20260807 19:41:48.301578 32237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:48 dist-test-slave-vq2q krb5kdc[32121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131708, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:48.304593 32237 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:41:48.306028 32237 webserver.cc:460] Webserver started at http://127.30.208.1:44535/ using document root <none> and password file <none>
I20260807 19:41:48.306644 32237 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:48.306746 32237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:48.307011 32237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:48.309890 32237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "e2fa4133d1434ad18e47a78b086e582b"
format_stamp: "Formatted at 2026-08-07 19:41:48 on dist-test-slave-vq2q"
server_key: "d3bd8040a19bb4b4e6a448cdefa52d26"
server_key_iv: "582c0d69b1379527d567669b91f8adb5"
server_key_version: "encryptionkey@0"
I20260807 19:41:48.310597 32237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "e2fa4133d1434ad18e47a78b086e582b"
format_stamp: "Formatted at 2026-08-07 19:41:48 on dist-test-slave-vq2q"
server_key: "d3bd8040a19bb4b4e6a448cdefa52d26"
server_key_iv: "582c0d69b1379527d567669b91f8adb5"
server_key_version: "encryptionkey@0"
I20260807 19:41:48.315467 32237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 19:41:48.318812 32255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:48.320120 32237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:41:48.320261 32237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "e2fa4133d1434ad18e47a78b086e582b"
format_stamp: "Formatted at 2026-08-07 19:41:48 on dist-test-slave-vq2q"
server_key: "d3bd8040a19bb4b4e6a448cdefa52d26"
server_key_iv: "582c0d69b1379527d567669b91f8adb5"
server_key_version: "encryptionkey@0"
I20260807 19:41:48.320456 32237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:48.356626 32237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:48.360991 32237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:48.361305 32237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:48.362300 32237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:41:48.363652 32237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:48.363724 32237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:48.363847 32237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:48.363907 32237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:48.394647 32237 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:37163
I20260807 19:41:48.394737 32372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:37163 every 8 connection(s)
I20260807 19:41:48.395896 32237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:41:48.400861 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 32237
I20260807 19:41:48.401052 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:41:48.401677 31552 external_mini_cluster.cc:1428] Setting key f997aa6a8bb19e9ecc8e62e7c58f070c
Aug 07 19:41:48 dist-test-slave-vq2q krb5kdc[32121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131708, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:48.415795 32375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:48.398333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:50163 (local address 127.30.208.62:45951)
0807 19:41:48.398779 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:48.398784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:48.399616 (+ 832us) server_negotiation.cc:407] Connection header received
0807 19:41:48.401609 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:48.401616 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:48.401710 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:48.401803 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:48.403417 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:48.404407 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:48.405991 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:48.406539 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:48.406669 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:48.410339 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:48.410344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:48.410347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:48.410379 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:48.412801 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:48.413569 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:48.413573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:48.413575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:48.413692 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:48.414096 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:48.414100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:48.414102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:48.414363 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:48.414497 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:48.415199 (+ 702us) server_negotiation.cc:299] Negotiation successful
0807 19:41:48.415436 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":225,"threads_started":1}
I20260807 19:41:48.416730 32376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:48.398785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:45951 (local address 127.30.208.1:50163)
0807 19:41:48.399463 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:48.399500 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:48.401356 (+ 1856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:48.402127 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:48.402138 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:48.402712 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:48.403264 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:48.403281 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:48.404572 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:48.404576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:48.405839 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:48.405853 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:48.406730 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:48.406734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:48.406971 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:48.407577 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:48.407604 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:48.410083 (+ 2479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:48.412935 (+ 2852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:48.412944 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:48.412960 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:48.413449 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:48.413793 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:48.413798 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:48.413801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:48.413982 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:48.414488 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:48.414496 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:48.414816 (+ 320us) client_negotiation.cc:769] Sending connection context
0807 19:41:48.415054 (+ 238us) client_negotiation.cc:240] Negotiation successful
0807 19:41:48.415585 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":263,"threads_started":1}
I20260807 19:41:48.418406 32373 heartbeater.cc:347] Connected to a master server at 127.30.208.62:45951
I20260807 19:41:48.418694 32373 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
I20260807 19:41:48.419414 32373 heartbeater.cc:511] Master 127.30.208.62:45951 requested a full tablet report, sending...
I20260807 19:41:48.421669 32172 ts_manager.cc:194] Registered new tserver with Master: e2fa4133d1434ad18e47a78b086e582b (127.30.208.1:37163)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
I20260807 19:41:48.423123 32172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:50163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:41:48.444044 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45951
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:38611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:48.766280 32379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:48.766609 32379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:48.766768 32379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:48.774525 32379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:48.774637 32379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:48.774763 32379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:41:48.784500 32379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:38611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45951
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:48.785837 32379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:48.787173 32379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:48.799743 32385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:48.799762 32386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:48.800863 32379 server_base.cc:1034] running on GCE node
W20260807 19:41:48.805967 32388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:48.806659 32379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:48.807639 32379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:48.808887 32379 hybrid_clock.cc:648] HybridClock initialized: now 1786131708808811 us; error 60 us; skew 500 ppm
I20260807 19:41:48.809239 32379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:48 dist-test-slave-vq2q krb5kdc[32121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131708, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:48.812448 32379 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:41:48.814016 32379 webserver.cc:460] Webserver started at http://127.30.208.2:46323/ using document root <none> and password file <none>
I20260807 19:41:48.814714 32379 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:48.814826 32379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:48.815119 32379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:48.818126 32379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "60608c9f30c14340806fadc3268a4c57"
format_stamp: "Formatted at 2026-08-07 19:41:48 on dist-test-slave-vq2q"
server_key: "ada980fb8fc44c58c348f54c9fa9deac"
server_key_iv: "0358540c3d7484cb6dbba66ff2b57e95"
server_key_version: "encryptionkey@0"
I20260807 19:41:48.818984 32379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "60608c9f30c14340806fadc3268a4c57"
format_stamp: "Formatted at 2026-08-07 19:41:48 on dist-test-slave-vq2q"
server_key: "ada980fb8fc44c58c348f54c9fa9deac"
server_key_iv: "0358540c3d7484cb6dbba66ff2b57e95"
server_key_version: "encryptionkey@0"
I20260807 19:41:48.824503 32379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:41:48.828193 32395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:48.829645 32379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:41:48.829841 32379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "60608c9f30c14340806fadc3268a4c57"
format_stamp: "Formatted at 2026-08-07 19:41:48 on dist-test-slave-vq2q"
server_key: "ada980fb8fc44c58c348f54c9fa9deac"
server_key_iv: "0358540c3d7484cb6dbba66ff2b57e95"
server_key_version: "encryptionkey@0"
I20260807 19:41:48.830037 32379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:48.846696 32379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:48.851104 32379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:48.851394 32379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:48.852453 32379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:41:48.853843 32379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:48.853916 32379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:48.854035 32379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:48.854094 32379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:48.885568 32379 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:39237
I20260807 19:41:48.885618 32511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:39237 every 8 connection(s)
I20260807 19:41:48.886922 32379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:41:48.892383 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 32379
I20260807 19:41:48.892601 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:41:48.893158 31552 external_mini_cluster.cc:1428] Setting key 8783aad1a5ee6672e962df66b583f486
Aug 07 19:41:48 dist-test-slave-vq2q krb5kdc[32121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131708, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:48.908699 32375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:48.889368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:38627 (local address 127.30.208.62:45951)
0807 19:41:48.889584 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:48.889591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:48.890840 (+ 1249us) server_negotiation.cc:407] Connection header received
0807 19:41:48.892920 (+ 2080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:48.892925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:48.893018 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:48.893111 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:48.894847 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:48.895679 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:48.897975 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:48.898412 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:48.898509 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:48.902411 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:48.902418 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:48.902423 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:48.902467 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:48.905590 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:48.906314 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:48.906319 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:48.906322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:48.906476 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:48.906911 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:48.906915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:48.906918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:48.907259 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:48.907398 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:48.908144 (+ 746us) server_negotiation.cc:299] Negotiation successful
0807 19:41:48.908364 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:41:48.909627 32514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:48.889933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:45951 (local address 127.30.208.2:38627)
0807 19:41:48.890659 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:48.890704 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:48.892608 (+ 1904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:48.893427 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:48.893438 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:48.894063 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:48.894637 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:48.894654 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:48.895856 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:48.895862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:48.897824 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:48.897840 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:48.898641 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:48.898648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:48.898929 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:48.899619 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:48.899647 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:48.902138 (+ 2491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:48.905741 (+ 3603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:48.905748 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:48.905760 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:48.906181 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:48.906570 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:48.906573 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:48.906575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:48.906784 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:48.907373 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:48.907399 (+ 26us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:48.907727 (+ 328us) client_negotiation.cc:769] Sending connection context
0807 19:41:48.908010 (+ 283us) client_negotiation.cc:240] Negotiation successful
0807 19:41:48.908412 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":288,"threads_started":1}
I20260807 19:41:48.911142 32512 heartbeater.cc:347] Connected to a master server at 127.30.208.62:45951
I20260807 19:41:48.911437 32512 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:48.912150 32512 heartbeater.cc:511] Master 127.30.208.62:45951 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
I20260807 19:41:48.913641 32172 ts_manager.cc:194] Registered new tserver with Master: 60608c9f30c14340806fadc3268a4c57 (127.30.208.2:39237)
I20260807 19:41:48.914412 32172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:38627
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:41:48.938870 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45951
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:38611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:49.270931 32517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:49.271214 32517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:49.271312 32517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:49.278555 32517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:49.278672 32517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:49.278800 32517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:41:49.288906 32517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:38611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45951
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:49.290238 32517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:49.291522 32517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:49.304625 32522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:49.304644 32523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:49.305240 32525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:49.305518 32517 server_base.cc:1034] running on GCE node
I20260807 19:41:49.306175 32517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:49.307098 32517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:49.308306 32517 hybrid_clock.cc:648] HybridClock initialized: now 1786131709308257 us; error 38 us; skew 500 ppm
I20260807 19:41:49.308589 32517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:49 dist-test-slave-vq2q krb5kdc[32121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131709, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:49.311674 32517 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:41:49.313206 32517 webserver.cc:460] Webserver started at http://127.30.208.3:35169/ using document root <none> and password file <none>
I20260807 19:41:49.313894 32517 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:49.314018 32517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:49.314313 32517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:49.317121 32517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "924824d894934bf69727d23923a13755"
format_stamp: "Formatted at 2026-08-07 19:41:49 on dist-test-slave-vq2q"
server_key: "3c30ab1dde1cf5cadc453866fb0d9eea"
server_key_iv: "a7e831e924326f8372a56cc072324616"
server_key_version: "encryptionkey@0"
I20260807 19:41:49.318035 32517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "924824d894934bf69727d23923a13755"
format_stamp: "Formatted at 2026-08-07 19:41:49 on dist-test-slave-vq2q"
server_key: "3c30ab1dde1cf5cadc453866fb0d9eea"
server_key_iv: "a7e831e924326f8372a56cc072324616"
server_key_version: "encryptionkey@0"
I20260807 19:41:49.323022 32517 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:41:49.326494 32532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:49.327941 32517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:41:49.328102 32517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "924824d894934bf69727d23923a13755"
format_stamp: "Formatted at 2026-08-07 19:41:49 on dist-test-slave-vq2q"
server_key: "3c30ab1dde1cf5cadc453866fb0d9eea"
server_key_iv: "a7e831e924326f8372a56cc072324616"
server_key_version: "encryptionkey@0"
I20260807 19:41:49.328285 32517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:49.352881 32517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:49.356985 32517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:49.357318 32517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:49.358419 32517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:41:49.359831 32517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:49.359912 32517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:49.360029 32517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:49.360083 32517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:49.392253 32517 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:41773
I20260807 19:41:49.392320 32654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:41773 every 8 connection(s)
I20260807 19:41:49.393548 32517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:41:49.398303 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 32517
I20260807 19:41:49.398492 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:41:49.398893 31552 external_mini_cluster.cc:1428] Setting key 161a8137f436dfe0f66f124cd127b4c0
Aug 07 19:41:49 dist-test-slave-vq2q krb5kdc[32121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131709, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:49.414108 32375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.396376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:50923 (local address 127.30.208.62:45951)
0807 19:41:49.396546 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:49.396550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:49.397972 (+ 1422us) server_negotiation.cc:407] Connection header received
0807 19:41:49.400216 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:49.400222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:49.400345 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:49.400452 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:49.402403 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.403202 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.404666 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.405087 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.405209 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.408392 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:49.408396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:49.408399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:49.408431 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:49.411106 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:49.411826 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:49.411829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:49.411831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:49.411953 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:49.412323 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:49.412327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:49.412330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:49.412643 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:49.412800 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:49.413556 (+ 756us) server_negotiation.cc:299] Negotiation successful
0807 19:41:49.413796 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:41:49.415077 32657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.396962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:45951 (local address 127.30.208.3:50923)
0807 19:41:49.397787 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:49.397845 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:49.399871 (+ 2026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:49.400723 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:49.400736 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:49.401525 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:49.402241 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:49.402262 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.403346 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.403350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:49.404553 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:49.404566 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.405292 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.405298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:49.405484 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.406093 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:49.406114 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:49.408206 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:49.411254 (+ 3048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:49.411261 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:49.411286 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:49.411725 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:49.412033 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:49.412036 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:49.412039 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:49.412217 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:49.412770 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:49.412777 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:49.413084 (+ 307us) client_negotiation.cc:769] Sending connection context
0807 19:41:49.413406 (+ 322us) client_negotiation.cc:240] Negotiation successful
0807 19:41:49.413854 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":365,"threads_started":1}
I20260807 19:41:49.416334 32655 heartbeater.cc:347] Connected to a master server at 127.30.208.62:45951
I20260807 19:41:49.416615 32655 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:49.417362 32655 heartbeater.cc:511] Master 127.30.208.62:45951 requested a full tablet report, sending...
I20260807 19:41:49.418815 32172 ts_manager.cc:194] Registered new tserver with Master: 924824d894934bf69727d23923a13755 (127.30.208.3:41773)
I20260807 19:41:49.419610 32172 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:50923
I20260807 19:41:49.426213 32373 heartbeater.cc:503] Master 127.30.208.62:45951 was elected leader, sending a full tablet report...
I20260807 19:41:49.427335 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:41:49.445299 32375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.433543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42924 (local address 127.30.208.62:45951)
0807 19:41:49.433720 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:49.433726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:49.434236 (+ 510us) server_negotiation.cc:407] Connection header received
0807 19:41:49.434382 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:49.434386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:49.434451 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:49.434535 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:49.435983 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.436704 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.437998 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.438388 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.438475 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.440651 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:49.440656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:49.440659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:49.440692 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:49.443098 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:49.443709 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:49.443713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:49.443715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:49.443840 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:49.444114 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:49.444117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:49.444119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:49.444426 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:49.444613 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:49.444854 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 19:41:49.445022 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 19:41:49.451628 32171 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:41:49.458699 32171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:41:49.481642 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.471938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50568 (local address 127.30.208.3:41773)
0807 19:41:49.473401 (+ 1463us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:49.473409 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:49.473943 (+ 534us) server_negotiation.cc:407] Connection header received
0807 19:41:49.474048 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:49.474055 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:49.474274 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:49.474445 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:49.475587 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.477100 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.479579 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.480511 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.480637 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.480879 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:49.481111 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 19:41:49.481310 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1305,"thread_start_us":1091,"threads_started":1}
I20260807 19:41:49.481717 32671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.471493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:41773 (local address 127.0.0.1:50568)
0807 19:41:49.473483 (+ 1990us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:49.473527 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:49.473742 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:49.474740 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:49.474745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:49.474771 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:49.475334 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:49.475352 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.477251 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.477257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:49.479448 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:49.479458 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.480699 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.480706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:49.480894 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.480944 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 19:41:49.481056 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 19:41:49.481284 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1745,"mutex_wait_us":967,"thread_start_us":642,"threads_started":1}
I20260807 19:41:49.482198 32672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.471985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:39237 (local address 127.0.0.1:44852)
0807 19:41:49.473861 (+ 1876us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:49.473877 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:49.474069 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:49.474883 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:49.474888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:49.474917 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:49.475497 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:49.475510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.476837 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.476841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:49.479221 (+ 2380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:49.479236 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.481226 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.481232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:49.481419 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.481519 (+ 100us) client_negotiation.cc:769] Sending connection context
0807 19:41:49.481615 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 19:41:49.481756 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1745,"mutex_wait_us":756,"thread_start_us":957,"threads_started":1}
I20260807 19:41:49.482196 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.471077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49634 (local address 127.30.208.1:37163)
0807 19:41:49.472500 (+ 1423us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:49.472509 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:49.473649 (+ 1140us) server_negotiation.cc:407] Connection header received
0807 19:41:49.473958 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:49.473965 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:49.474184 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:49.474400 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:49.475148 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.476857 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.479280 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.480737 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.480857 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.481074 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:49.481716 (+ 642us) server_negotiation.cc:299] Negotiation successful
0807 19:41:49.481882 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1242,"thread_start_us":1293,"threads_started":1}
I20260807 19:41:49.482808 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.472255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44852 (local address 127.30.208.2:39237)
0807 19:41:49.472921 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:49.472929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:49.473962 (+ 1033us) server_negotiation.cc:407] Connection header received
0807 19:41:49.474245 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:49.474253 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:49.474464 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:49.474640 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:49.475680 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.476687 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.479396 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.480737 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.480878 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.481035 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:49.482213 (+ 1178us) server_negotiation.cc:299] Negotiation successful
0807 19:41:49.482520 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":446,"threads_started":1}
I20260807 19:41:49.483042 32667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.470785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:37163 (local address 127.0.0.1:49634)
0807 19:41:49.473483 (+ 2698us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:49.473527 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:49.473765 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:49.474416 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:49.474423 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:49.474454 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:49.474918 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:49.474931 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.477038 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.477043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:49.479091 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:49.479107 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.481283 (+ 2176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.481290 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:49.481520 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.481546 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:41:49.482542 (+ 996us) client_negotiation.cc:240] Negotiation successful
0807 19:41:49.482672 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1599,"mutex_wait_us":48,"thread_start_us":366,"threads_started":1}
I20260807 19:41:49.485793 32583 tablet_service.cc:1467] Processing CreateTablet for tablet 7f3c0da6b0534137a77ed0fadaad68d0 (DEFAULT_TABLE table=test-table [id=5708b0fcf0724d29b56005e6e4384420]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:49.486403 32305 tablet_service.cc:1467] Processing CreateTablet for tablet 7f3c0da6b0534137a77ed0fadaad68d0 (DEFAULT_TABLE table=test-table [id=5708b0fcf0724d29b56005e6e4384420]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:49.486397 32445 tablet_service.cc:1467] Processing CreateTablet for tablet 7f3c0da6b0534137a77ed0fadaad68d0 (DEFAULT_TABLE table=test-table [id=5708b0fcf0724d29b56005e6e4384420]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:49.487131 32583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f3c0da6b0534137a77ed0fadaad68d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:49.487728 32305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f3c0da6b0534137a77ed0fadaad68d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:49.487754 32445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f3c0da6b0534137a77ed0fadaad68d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:49.499717 32673 tablet_bootstrap.cc:492] T 7f3c0da6b0534137a77ed0fadaad68d0 P 924824d894934bf69727d23923a13755: Bootstrap starting.
I20260807 19:41:49.503376 32673 tablet_bootstrap.cc:654] T 7f3c0da6b0534137a77ed0fadaad68d0 P 924824d894934bf69727d23923a13755: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:49.504429 32673 log.cc:824] T 7f3c0da6b0534137a77ed0fadaad68d0 P 924824d894934bf69727d23923a13755: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:49.506891 32675 tablet_bootstrap.cc:492] T 7f3c0da6b0534137a77ed0fadaad68d0 P 60608c9f30c14340806fadc3268a4c57: Bootstrap starting.
I20260807 19:41:49.507587 32673 tablet_bootstrap.cc:492] T 7f3c0da6b0534137a77ed0fadaad68d0 P 924824d894934bf69727d23923a13755: No bootstrap required, opened a new log
I20260807 19:41:49.507856 32673 ts_tablet_manager.cc:1397] T 7f3c0da6b0534137a77ed0fadaad68d0 P 924824d894934bf69727d23923a13755: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260807 19:41:49.505851 32674 tablet_bootstrap.cc:492] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b: Bootstrap starting.
I20260807 19:41:49.509997 32675 tablet_bootstrap.cc:654] T 7f3c0da6b0534137a77ed0fadaad68d0 P 60608c9f30c14340806fadc3268a4c57: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:49.511327 32675 log.cc:824] T 7f3c0da6b0534137a77ed0fadaad68d0 P 60608c9f30c14340806fadc3268a4c57: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:49.512647 32674 tablet_bootstrap.cc:654] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:49.513991 32674 log.cc:824] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:49.514793 32675 tablet_bootstrap.cc:492] T 7f3c0da6b0534137a77ed0fadaad68d0 P 60608c9f30c14340806fadc3268a4c57: No bootstrap required, opened a new log
I20260807 19:41:49.515046 32675 ts_tablet_manager.cc:1397] T 7f3c0da6b0534137a77ed0fadaad68d0 P 60608c9f30c14340806fadc3268a4c57: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:41:49.516278 32673 raft_consensus.cc:348] T 7f3c0da6b0534137a77ed0fadaad68d0 P 924824d894934bf69727d23923a13755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fa4133d1434ad18e47a78b086e582b" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37163 } } peers { permanent_uuid: "60608c9f30c14340806fadc3268a4c57" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39237 } } peers { permanent_uuid: "924824d894934bf69727d23923a13755" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41773 } }
I20260807 19:41:49.516635 32673 raft_consensus.cc:374] T 7f3c0da6b0534137a77ed0fadaad68d0 P 924824d894934bf69727d23923a13755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:49.516741 32673 raft_consensus.cc:729] T 7f3c0da6b0534137a77ed0fadaad68d0 P 924824d894934bf69727d23923a13755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 924824d894934bf69727d23923a13755, State: Initialized, Role: FOLLOWER
I20260807 19:41:49.517120 32674 tablet_bootstrap.cc:492] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b: No bootstrap required, opened a new log
I20260807 19:41:49.517431 32674 ts_tablet_manager.cc:1397] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.005s
I20260807 19:41:49.517364 32673 consensus_queue.cc:260] T 7f3c0da6b0534137a77ed0fadaad68d0 P 924824d894934bf69727d23923a13755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fa4133d1434ad18e47a78b086e582b" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37163 } } peers { permanent_uuid: "60608c9f30c14340806fadc3268a4c57" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39237 } } peers { permanent_uuid: "924824d894934bf69727d23923a13755" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41773 } }
I20260807 19:41:49.523648 32655 heartbeater.cc:503] Master 127.30.208.62:45951 was elected leader, sending a full tablet report...
I20260807 19:41:49.524171 32673 ts_tablet_manager.cc:1428] T 7f3c0da6b0534137a77ed0fadaad68d0 P 924824d894934bf69727d23923a13755: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260807 19:41:49.526186 32675 raft_consensus.cc:348] T 7f3c0da6b0534137a77ed0fadaad68d0 P 60608c9f30c14340806fadc3268a4c57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fa4133d1434ad18e47a78b086e582b" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37163 } } peers { permanent_uuid: "60608c9f30c14340806fadc3268a4c57" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39237 } } peers { permanent_uuid: "924824d894934bf69727d23923a13755" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41773 } }
I20260807 19:41:49.526674 32675 raft_consensus.cc:374] T 7f3c0da6b0534137a77ed0fadaad68d0 P 60608c9f30c14340806fadc3268a4c57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:49.526784 32675 raft_consensus.cc:729] T 7f3c0da6b0534137a77ed0fadaad68d0 P 60608c9f30c14340806fadc3268a4c57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60608c9f30c14340806fadc3268a4c57, State: Initialized, Role: FOLLOWER
I20260807 19:41:49.527117 32674 raft_consensus.cc:348] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fa4133d1434ad18e47a78b086e582b" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37163 } } peers { permanent_uuid: "60608c9f30c14340806fadc3268a4c57" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39237 } } peers { permanent_uuid: "924824d894934bf69727d23923a13755" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41773 } }
I20260807 19:41:49.527544 32674 raft_consensus.cc:374] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:49.527436 32675 consensus_queue.cc:260] T 7f3c0da6b0534137a77ed0fadaad68d0 P 60608c9f30c14340806fadc3268a4c57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fa4133d1434ad18e47a78b086e582b" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37163 } } peers { permanent_uuid: "60608c9f30c14340806fadc3268a4c57" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39237 } } peers { permanent_uuid: "924824d894934bf69727d23923a13755" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41773 } }
I20260807 19:41:49.527630 32674 raft_consensus.cc:729] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2fa4133d1434ad18e47a78b086e582b, State: Initialized, Role: FOLLOWER
I20260807 19:41:49.528187 32674 consensus_queue.cc:260] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fa4133d1434ad18e47a78b086e582b" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37163 } } peers { permanent_uuid: "60608c9f30c14340806fadc3268a4c57" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39237 } } peers { permanent_uuid: "924824d894934bf69727d23923a13755" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41773 } }
I20260807 19:41:49.529086 32512 heartbeater.cc:503] Master 127.30.208.62:45951 was elected leader, sending a full tablet report...
I20260807 19:41:49.530005 32675 ts_tablet_manager.cc:1428] T 7f3c0da6b0534137a77ed0fadaad68d0 P 60608c9f30c14340806fadc3268a4c57: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260807 19:41:49.531486 32674 ts_tablet_manager.cc:1428] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
W20260807 19:41:49.638952 32513 tablet.cc:2369] T 7f3c0da6b0534137a77ed0fadaad68d0 P 60608c9f30c14340806fadc3268a4c57: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:41:49.645346 32656 tablet.cc:2369] T 7f3c0da6b0534137a77ed0fadaad68d0 P 924824d894934bf69727d23923a13755: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:41:49.648255 32374 tablet.cc:2369] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:41:49.653895 32682 raft_consensus.cc:482] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:41:49.654214 32682 raft_consensus.cc:504] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fa4133d1434ad18e47a78b086e582b" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37163 } } peers { permanent_uuid: "60608c9f30c14340806fadc3268a4c57" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39237 } } peers { permanent_uuid: "924824d894934bf69727d23923a13755" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41773 } }
I20260807 19:41:49.655972 32682 leader_election.cc:288] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60608c9f30c14340806fadc3268a4c57 (127.30.208.2:39237), 924824d894934bf69727d23923a13755 (127.30.208.3:41773)
I20260807 19:41:49.664224 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.656615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:52851 (local address 127.30.208.3:41773)
0807 19:41:49.656839 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:49.656846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:49.657534 (+ 688us) server_negotiation.cc:407] Connection header received
0807 19:41:49.657676 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:49.657680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:49.657741 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:49.657827 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:49.658520 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.659599 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.662408 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.663326 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.663471 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.663545 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:49.663845 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 19:41:49.663982 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:41:49.664270 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.656456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:52359 (local address 127.30.208.2:39237)
0807 19:41:49.656683 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:49.656690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:49.657440 (+ 750us) server_negotiation.cc:407] Connection header received
0807 19:41:49.657623 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:49.657629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:49.657704 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:49.657774 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:49.658416 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.659531 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.662404 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.663364 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.663519 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.663591 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:49.663901 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 19:41:49.664027 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:41:49.664403 32685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.656391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:41773 (local address 127.30.208.1:52851)
0807 19:41:49.657401 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:49.657421 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:49.657561 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:49.657863 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:49.657868 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:49.657892 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:49.658366 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:49.658378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.659731 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.659736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:49.662215 (+ 2479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:49.662230 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.663549 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.663557 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:49.663693 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.663720 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 19:41:49.663804 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 19:41:49.663932 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"mutex_wait_us":79,"thread_start_us":662,"threads_started":1}
I20260807 19:41:49.664465 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.656212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:39237 (local address 127.30.208.1:52359)
0807 19:41:49.657316 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:49.657333 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:49.657472 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:49.657816 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:49.657821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:49.657844 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:49.658259 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:49.658269 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.659659 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.659664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:49.662225 (+ 2561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:49.662235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.663600 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.663606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:49.663756 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.663779 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:41:49.663855 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 19:41:49.663990 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":981,"mutex_wait_us":105,"thread_start_us":702,"threads_started":1}
I20260807 19:41:49.665496 32465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3c0da6b0534137a77ed0fadaad68d0" candidate_uuid: "e2fa4133d1434ad18e47a78b086e582b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60608c9f30c14340806fadc3268a4c57" is_pre_election: true
I20260807 19:41:49.665599 32606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3c0da6b0534137a77ed0fadaad68d0" candidate_uuid: "e2fa4133d1434ad18e47a78b086e582b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "924824d894934bf69727d23923a13755" is_pre_election: true
I20260807 19:41:49.666035 32465 raft_consensus.cc:2454] T 7f3c0da6b0534137a77ed0fadaad68d0 P 60608c9f30c14340806fadc3268a4c57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2fa4133d1434ad18e47a78b086e582b in term 0.
I20260807 19:41:49.666071 32606 raft_consensus.cc:2454] T 7f3c0da6b0534137a77ed0fadaad68d0 P 924824d894934bf69727d23923a13755 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2fa4133d1434ad18e47a78b086e582b in term 0.
I20260807 19:41:49.666668 32257 leader_election.cc:302] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60608c9f30c14340806fadc3268a4c57, e2fa4133d1434ad18e47a78b086e582b; no voters:
I20260807 19:41:49.667248 32682 raft_consensus.cc:2781] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:41:49.667440 32682 raft_consensus.cc:482] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:41:49.667542 32682 raft_consensus.cc:3037] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:49.669780 32682 raft_consensus.cc:504] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fa4133d1434ad18e47a78b086e582b" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37163 } } peers { permanent_uuid: "60608c9f30c14340806fadc3268a4c57" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39237 } } peers { permanent_uuid: "924824d894934bf69727d23923a13755" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41773 } }
I20260807 19:41:49.670387 32682 leader_election.cc:288] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [CANDIDATE]: Term 1 election: Requested vote from peers 60608c9f30c14340806fadc3268a4c57 (127.30.208.2:39237), 924824d894934bf69727d23923a13755 (127.30.208.3:41773)
I20260807 19:41:49.670893 32606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3c0da6b0534137a77ed0fadaad68d0" candidate_uuid: "e2fa4133d1434ad18e47a78b086e582b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "924824d894934bf69727d23923a13755"
I20260807 19:41:49.670881 32465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f3c0da6b0534137a77ed0fadaad68d0" candidate_uuid: "e2fa4133d1434ad18e47a78b086e582b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60608c9f30c14340806fadc3268a4c57"
I20260807 19:41:49.671108 32606 raft_consensus.cc:3037] T 7f3c0da6b0534137a77ed0fadaad68d0 P 924824d894934bf69727d23923a13755 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:49.671106 32465 raft_consensus.cc:3037] T 7f3c0da6b0534137a77ed0fadaad68d0 P 60608c9f30c14340806fadc3268a4c57 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:49.674389 32606 raft_consensus.cc:2454] T 7f3c0da6b0534137a77ed0fadaad68d0 P 924824d894934bf69727d23923a13755 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2fa4133d1434ad18e47a78b086e582b in term 1.
I20260807 19:41:49.674383 32465 raft_consensus.cc:2454] T 7f3c0da6b0534137a77ed0fadaad68d0 P 60608c9f30c14340806fadc3268a4c57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2fa4133d1434ad18e47a78b086e582b in term 1.
I20260807 19:41:49.674901 32256 leader_election.cc:302] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 924824d894934bf69727d23923a13755, e2fa4133d1434ad18e47a78b086e582b; no voters:
I20260807 19:41:49.675220 32682 raft_consensus.cc:2781] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:41:49.675688 32682 raft_consensus.cc:686] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [term 1 LEADER]: Becoming Leader. State: Replica: e2fa4133d1434ad18e47a78b086e582b, State: Running, Role: LEADER
I20260807 19:41:49.676232 32682 consensus_queue.cc:237] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fa4133d1434ad18e47a78b086e582b" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37163 } } peers { permanent_uuid: "60608c9f30c14340806fadc3268a4c57" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39237 } } peers { permanent_uuid: "924824d894934bf69727d23923a13755" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41773 } }
I20260807 19:41:49.680824 32171 catalog_manager.cc:5495] T 7f3c0da6b0534137a77ed0fadaad68d0 P e2fa4133d1434ad18e47a78b086e582b reported cstate change: term changed from 0 to 1, leader changed from <none> to e2fa4133d1434ad18e47a78b086e582b (127.30.208.1). New cstate: current_term: 1 leader_uuid: "e2fa4133d1434ad18e47a78b086e582b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fa4133d1434ad18e47a78b086e582b" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60608c9f30c14340806fadc3268a4c57" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "924824d894934bf69727d23923a13755" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41773 } health_report { overall_health: UNKNOWN } } }
Aug 07 19:41:49 dist-test-slave-vq2q krb5kdc[32121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:41:49.726850 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.711725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.30.208.1:37163)
0807 19:41:49.711923 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:49.711928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:49.712417 (+ 489us) server_negotiation.cc:407] Connection header received
0807 19:41:49.712589 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:49.712593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:49.712660 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:49.712745 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:49.714372 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.715080 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.716336 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.716760 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.716873 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.720977 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:49.720995 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:49.721011 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:49.721051 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:49.723961 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:49.724647 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:49.724655 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:49.724658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:49.724818 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:49.725276 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:49.725281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:49.725284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:49.725657 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:49.725968 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:49.726289 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 19:41:49.726530 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 19:41:49 dist-test-slave-vq2q krb5kdc[32121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131709, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:41:49 dist-test-slave-vq2q krb5kdc[32121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131709, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:41:49.777963 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.763043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41584 (local address 127.30.208.1:37163)
0807 19:41:49.763246 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:49.763250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:49.763708 (+ 458us) server_negotiation.cc:407] Connection header received
0807 19:41:49.763926 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:49.763931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:49.763999 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:49.764075 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:49.765593 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.766502 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.767777 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.768181 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.768269 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.772041 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:49.772046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:49.772050 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:49.772089 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:49.774902 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:49.775735 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:49.775741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:49.775744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:49.775920 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:49.776342 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:49.776347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:49.776351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:49.776691 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:49.776991 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:49.777342 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 19:41:49.777590 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 19:41:49 dist-test-slave-vq2q krb5kdc[32121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:41:49 dist-test-slave-vq2q krb5kdc[32121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:41:49.828759 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.813291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41592 (local address 127.30.208.1:37163)
0807 19:41:49.813502 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:49.813508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:49.813830 (+ 322us) server_negotiation.cc:407] Connection header received
0807 19:41:49.814035 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:49.814039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:49.814127 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:49.814210 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:49.815611 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.816365 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.817608 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.818046 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.818159 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.822843 (+ 4684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:49.822849 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:49.822852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:49.822916 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:49.825779 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:49.826589 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:49.826596 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:49.826598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:49.826782 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:49.827218 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:49.827222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:49.827224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:49.827540 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:49.827811 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:49.828143 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 19:41:49.828412 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 19:41:49.830781 32283 tablet_service.cc:2908] Rejecting scan request for tablet 7f3c0da6b0534137a77ed0fadaad68d0: Uninitialized: clean time has not yet been initialized
I20260807 19:41:49.834072 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 32237
I20260807 19:41:49.847337 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.834927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41606 (local address 127.30.208.1:37163)
0807 19:41:49.835083 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:49.835087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:49.835532 (+ 445us) server_negotiation.cc:407] Connection header received
0807 19:41:49.835703 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:49.835709 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:49.835786 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:49.835871 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:49.837097 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.837904 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.839088 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.839506 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.839614 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.842012 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:49.842018 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:49.842021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:49.842055 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:49.844702 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:49.845444 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:49.845450 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:49.845453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:49.845590 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:49.845949 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:49.845953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:49.845955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:49.846212 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:49.846464 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:49.846741 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 19:41:49.846998 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 19:41:49.847996 32368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:49.941358 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 32237
I20260807 19:41:49.959090 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 32379
Aug 07 19:41:49 dist-test-slave-vq2q krb5kdc[32121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:41:49.973829 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:49.959969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47022 (local address 127.30.208.2:39237)
0807 19:41:49.960208 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:49.960213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:49.960232 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:41:49.960307 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:49.960311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:49.960378 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:49.960455 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:49.961654 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.962492 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.963762 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:49.964180 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:49.964304 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:49.968133 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:49.968150 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:49.968167 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:49.968205 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:49.970885 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:49.971690 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:49.971698 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:49.971701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:49.971854 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:49.972175 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:49.972178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:49.972180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:49.972605 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:49.972858 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:49.973272 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 19:41:49.973484 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 19:41:49.974491 32507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:50.064278 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 32379
I20260807 19:41:50.080744 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 32517
Aug 07 19:41:50 dist-test-slave-vq2q krb5kdc[32121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:41:50.095461 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:50.081517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38292 (local address 127.30.208.3:41773)
0807 19:41:50.081704 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:50.081710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:50.081732 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:41:50.081884 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:50.081888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:50.081963 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:50.082051 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:50.083283 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:50.084004 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:50.085414 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:50.085828 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:50.085941 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:50.090110 (+ 4169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:50.090126 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:50.090141 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:50.090176 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:50.092652 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:50.093410 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:50.093417 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:50.093420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:50.093581 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:50.093948 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:50.093951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:50.093954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:50.094320 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:50.094595 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:50.094914 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 19:41:50.095133 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:41:50.096122 32649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:50.184408 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 32517
I20260807 19:41:50.202291 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 32137
I20260807 19:41:50.203526 32203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:50.312351 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 32137
2026-08-07T19:41:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4246 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:41:50 dist-test-slave-vq2q krb5kdc[32707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:41:50 dist-test-slave-vq2q krb5kdc[32707](info): set up 2 sockets
Aug 07 19:41:50 dist-test-slave-vq2q krb5kdc[32707](info): commencing operation
krb5kdc: starting...
W20260807 19:41:51.383574 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:41:51 dist-test-slave-vq2q krb5kdc[32707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:41:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:41:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:41:51.576875 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:35625
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:35625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:35625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:51.908593 32721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:51.908890 32721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:51.909006 32721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:51.916666 32721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:41:51.916795 32721 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 19:41:51.916846 32721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:51.916908 32721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:51.916957 32721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:41:51.917017 32721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:41:51.927259 32721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:35625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:35625
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:35625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:51.928576 32721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:51.929951 32721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:51.941594 32727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:51.941532 32730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:51.941617 32728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:51.942508 32721 server_base.cc:1034] running on GCE node
I20260807 19:41:51.943271 32721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:51.944713 32721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:51.945945 32721 hybrid_clock.cc:648] HybridClock initialized: now 1786131711945886 us; error 33 us; skew 500 ppm
I20260807 19:41:51.946285 32721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:51 dist-test-slave-vq2q krb5kdc[32707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131711, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:51.949522 32721 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:41:51.951086 32721 webserver.cc:460] Webserver started at http://127.30.208.62:41477/ using document root <none> and password file <none>
I20260807 19:41:51.951797 32721 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:51.951900 32721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:51.952221 32721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:51.955324 32721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "2f34dfa46cc64fad92a117ae135de749"
format_stamp: "Formatted at 2026-08-07 19:41:51 on dist-test-slave-vq2q"
server_key: "8a2b7d62b66b15509cfcb0f6800c8d65"
server_key_iv: "f62dab525e9a8bad6e0f62049fd107d5"
server_key_version: "encryptionkey@0"
I20260807 19:41:51.956079 32721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "2f34dfa46cc64fad92a117ae135de749"
format_stamp: "Formatted at 2026-08-07 19:41:51 on dist-test-slave-vq2q"
server_key: "8a2b7d62b66b15509cfcb0f6800c8d65"
server_key_iv: "f62dab525e9a8bad6e0f62049fd107d5"
server_key_version: "encryptionkey@0"
I20260807 19:41:51.961488 32721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 19:41:51.964877 32737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:51.966264 32721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:41:51.966408 32721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "2f34dfa46cc64fad92a117ae135de749"
format_stamp: "Formatted at 2026-08-07 19:41:51 on dist-test-slave-vq2q"
server_key: "8a2b7d62b66b15509cfcb0f6800c8d65"
server_key_iv: "f62dab525e9a8bad6e0f62049fd107d5"
server_key_version: "encryptionkey@0"
I20260807 19:41:51.966594 32721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:51.978166 32721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:51.982230 32721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:51.982470 32721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:52.003676 32721 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:35625
I20260807 19:41:52.003718 332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:35625 every 8 connection(s)
I20260807 19:41:52.005522 32721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:41:52.006134 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 32721
I20260807 19:41:52.006376 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:41:52.007143 31552 external_mini_cluster.cc:1428] Setting key a00157489c413f7ab6d69adcaa26a74f
I20260807 19:41:52.009965 333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:41:52 dist-test-slave-vq2q krb5kdc[32707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:52.020943 333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749: Bootstrap starting.
I20260807 19:41:52.024263 333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:52.025374 333 log.cc:824] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:52.028712 333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749: No bootstrap required, opened a new log
I20260807 19:41:52.029220 336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:52.009079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58096 (local address 127.30.208.62:35625)
0807 19:41:52.009826 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:52.009838 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:52.009884 (+ 46us) server_negotiation.cc:407] Connection header received
0807 19:41:52.011556 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:52.011579 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:52.011955 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:52.012330 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:52.013842 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:52.015269 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:52.016495 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:52.017220 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:52.017409 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:52.021729 (+ 4320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:52.021738 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:52.021753 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:52.021799 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:52.024860 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:52.025641 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:52.025648 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:52.025654 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:52.025843 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:52.026315 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:52.026320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:52.026323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:52.026981 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:52.027202 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:52.027663 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 19:41:52.028105 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":315,"threads_started":1}
I20260807 19:41:52.036966 333 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f34dfa46cc64fad92a117ae135de749" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 35625 } }
I20260807 19:41:52.037433 333 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:52.037531 333 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f34dfa46cc64fad92a117ae135de749, State: Initialized, Role: FOLLOWER
I20260807 19:41:52.038154 333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f34dfa46cc64fad92a117ae135de749" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 35625 } }
I20260807 19:41:52.038420 333 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:41:52.038599 333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:41:52.038748 333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:52.041208 333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f34dfa46cc64fad92a117ae135de749" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 35625 } }
I20260807 19:41:52.041718 333 leader_election.cc:302] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f34dfa46cc64fad92a117ae135de749; no voters:
I20260807 19:41:52.042274 333 leader_election.cc:288] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:41:52.042425 339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:41:52.043147 339 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [term 1 LEADER]: Becoming Leader. State: Replica: 2f34dfa46cc64fad92a117ae135de749, State: Running, Role: LEADER
I20260807 19:41:52.043655 339 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f34dfa46cc64fad92a117ae135de749" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 35625 } }
I20260807 19:41:52.044502 333 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:41:52.046567 340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f34dfa46cc64fad92a117ae135de749" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f34dfa46cc64fad92a117ae135de749" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 35625 } } }
I20260807 19:41:52.047014 340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [sys.catalog]: This master's current role is: LEADER
I20260807 19:41:52.047061 341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f34dfa46cc64fad92a117ae135de749. Latest consensus state: current_term: 1 leader_uuid: "2f34dfa46cc64fad92a117ae135de749" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f34dfa46cc64fad92a117ae135de749" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 35625 } } }
I20260807 19:41:52.047503 341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749 [sys.catalog]: This master's current role is: LEADER
I20260807 19:41:52.049235 348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:41:52.053916 348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:41:52.062052 348 catalog_manager.cc:1347] Generated new cluster ID: 8d0f98eed6234bf6966352b06e6a978f
I20260807 19:41:52.062175 348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:41:52.071761 348 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:41:52.073621 348 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:41:52.085793 348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749: Generated new TSK 0
I20260807 19:41:52.086918 348 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:41:52.137027 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:35625
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:35625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:52.467708 361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:52.467983 361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:52.468113 361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:52.475566 361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:52.475678 361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:52.475802 361 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:41:52.475909 361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:41:52.486181 361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:35625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.208.62:35625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:52.487562 361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:52.488969 361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:52.502023 368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:52.502130 367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:52.502162 370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:52.502918 361 server_base.cc:1034] running on GCE node
I20260807 19:41:52.503520 361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:52.504444 361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:52.505736 361 hybrid_clock.cc:648] HybridClock initialized: now 1786131712505645 us; error 32 us; skew 500 ppm
I20260807 19:41:52.506014 361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:52 dist-test-slave-vq2q krb5kdc[32707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131712, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:52.509021 361 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:41:52.510416 361 webserver.cc:460] Webserver started at http://127.30.208.1:39235/ using document root <none> and password file <none>
I20260807 19:41:52.511051 361 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:52.511137 361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:52.511396 361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:52.514243 361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "bca85bc96b104c22a05f824fdfcae087"
format_stamp: "Formatted at 2026-08-07 19:41:52 on dist-test-slave-vq2q"
server_key: "fe92da67d5f517b35d43d24a24d86ee5"
server_key_iv: "fe18075dc011de821c8973e23ad6a26c"
server_key_version: "encryptionkey@0"
I20260807 19:41:52.515002 361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "bca85bc96b104c22a05f824fdfcae087"
format_stamp: "Formatted at 2026-08-07 19:41:52 on dist-test-slave-vq2q"
server_key: "fe92da67d5f517b35d43d24a24d86ee5"
server_key_iv: "fe18075dc011de821c8973e23ad6a26c"
server_key_version: "encryptionkey@0"
I20260807 19:41:52.520072 361 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:41:52.523435 377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:52.524705 361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:41:52.524834 361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "bca85bc96b104c22a05f824fdfcae087"
format_stamp: "Formatted at 2026-08-07 19:41:52 on dist-test-slave-vq2q"
server_key: "fe92da67d5f517b35d43d24a24d86ee5"
server_key_iv: "fe18075dc011de821c8973e23ad6a26c"
server_key_version: "encryptionkey@0"
I20260807 19:41:52.525031 361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:52.538868 361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:52.542611 361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:52.542841 361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:52.545480 361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:52.545575 361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:52.545773 361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:52.545846 361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:41:52 dist-test-slave-vq2q krb5kdc[32707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131712, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:52.570973 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:52.546299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:59199 (local address 127.30.208.62:35625)
0807 19:41:52.546789 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:52.546796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:52.548193 (+ 1397us) server_negotiation.cc:407] Connection header received
0807 19:41:52.550984 (+ 2791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:52.550991 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:52.551099 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:52.551196 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:52.554239 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:52.555283 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:52.557847 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:52.558522 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:52.558660 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:52.563433 (+ 4773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:52.563440 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:52.563445 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:52.563494 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:52.566877 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:52.567895 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:52.567902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:52.567905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:52.568032 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:52.568647 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:52.568652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:52.568655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:52.569044 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:52.569220 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:52.570297 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0807 19:41:52.570562 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":241,"threads_started":1}
I20260807 19:41:52.572465 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:52.546879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:35625 (local address 127.30.208.1:59199)
0807 19:41:52.547733 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:52.547792 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:52.550565 (+ 2773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:52.551852 (+ 1287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:52.551864 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:52.552879 (+ 1015us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:52.553982 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:52.554018 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:52.555469 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:52.555475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:52.557639 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:52.557652 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:52.558799 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:52.558806 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:52.559031 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:52.559878 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:52.559903 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:52.563152 (+ 3249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:52.567068 (+ 3916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:52.567075 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:52.567088 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:52.567756 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:52.568150 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:52.568155 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:52.568159 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:52.568528 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:52.569290 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:52.569299 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:52.569815 (+ 516us) client_negotiation.cc:769] Sending connection context
0807 19:41:52.570161 (+ 346us) client_negotiation.cc:240] Negotiation successful
0807 19:41:52.570800 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":336,"threads_started":1}
I20260807 19:41:52.586316 361 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:42339
I20260807 19:41:52.586391 504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:42339 every 8 connection(s)
I20260807 19:41:52.587689 361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:41:52.589998 505 heartbeater.cc:347] Connected to a master server at 127.30.208.62:35625
I20260807 19:41:52.590346 505 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:52.591145 505 heartbeater.cc:511] Master 127.30.208.62:35625 requested a full tablet report, sending...
I20260807 19:41:52.593214 32754 ts_manager.cc:194] Registered new tserver with Master: bca85bc96b104c22a05f824fdfcae087 (127.30.208.1:42339)
I20260807 19:41:52.593516 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:52.579211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:54923 (local address 127.30.208.62:35625)
0807 19:41:52.579416 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:52.579423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:52.580245 (+ 822us) server_negotiation.cc:407] Connection header received
0807 19:41:52.580441 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:52.580446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:52.580519 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:52.580598 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:52.581922 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:52.582826 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:52.584627 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:52.585244 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:52.585377 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:52.587535 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:52.587541 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:52.587546 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:52.587583 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:52.590819 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:52.591505 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:52.591510 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:52.591514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:52.591674 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:52.592023 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:52.592028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:52.592032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:52.592367 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:52.592498 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:52.592906 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 19:41:52.593113 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 19:41:52.593554 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:52.579372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:35625 (local address 127.30.208.1:54923)
0807 19:41:52.580141 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:52.580162 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:52.580298 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:52.580636 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:52.580640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:52.581310 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:52.581782 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:52.581793 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:52.583004 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:52.583009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:52.584477 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:52.584490 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:52.585604 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:52.585608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:52.585753 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:52.586625 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:52.586646 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:52.587365 (+ 719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:52.590941 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:52.590944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:52.590947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:52.591395 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:52.591787 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:52.591790 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:52.591792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:52.591922 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:52.592481 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:52.592485 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:52.592653 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 19:41:52.592778 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 19:41:52.593079 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":447,"threads_started":1}
I20260807 19:41:52.595024 32754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:59199
I20260807 19:41:52.596612 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 361
I20260807 19:41:52.596836 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:41:52.597645 31552 external_mini_cluster.cc:1428] Setting key d4b8f04dffdf3d997769f8600ef244cf
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:41:52.638497 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:35625
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:35625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:52.964505 511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:52.964771 511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:52.964903 511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:52.972167 511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:52.972277 511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:52.972396 511 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:41:52.972486 511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:41:52.982560 511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:35625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.208.62:35625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:52.983908 511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:52.985309 511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:52.998330 517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:52.998394 516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:52.998369 519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:52.999782 511 server_base.cc:1034] running on GCE node
I20260807 19:41:53.000453 511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:53.001443 511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:53.002668 511 hybrid_clock.cc:648] HybridClock initialized: now 1786131713002597 us; error 36 us; skew 500 ppm
I20260807 19:41:53.003036 511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:53 dist-test-slave-vq2q krb5kdc[32707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131713, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:53.007051 511 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:41:53.018491 511 webserver.cc:460] Webserver started at http://127.30.208.2:35643/ using document root <none> and password file <none>
I20260807 19:41:53.019246 511 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:53.019359 511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:53.019680 511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:53.022666 511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "fa290f6d93644347af0c41cf053b09d3"
format_stamp: "Formatted at 2026-08-07 19:41:53 on dist-test-slave-vq2q"
server_key: "c9096bc2f7e1469b4e48afae85e57d58"
server_key_iv: "37aa0a99092d7513b69ac080910f5b0c"
server_key_version: "encryptionkey@0"
I20260807 19:41:53.023456 511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "fa290f6d93644347af0c41cf053b09d3"
format_stamp: "Formatted at 2026-08-07 19:41:53 on dist-test-slave-vq2q"
server_key: "c9096bc2f7e1469b4e48afae85e57d58"
server_key_iv: "37aa0a99092d7513b69ac080910f5b0c"
server_key_version: "encryptionkey@0"
I20260807 19:41:53.028748 511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:41:53.032791 526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:53.034126 511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:41:53.034291 511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "fa290f6d93644347af0c41cf053b09d3"
format_stamp: "Formatted at 2026-08-07 19:41:53 on dist-test-slave-vq2q"
server_key: "c9096bc2f7e1469b4e48afae85e57d58"
server_key_iv: "37aa0a99092d7513b69ac080910f5b0c"
server_key_version: "encryptionkey@0"
I20260807 19:41:53.034504 511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:53.051244 511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:53.055073 511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:53.055328 511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:53.058086 511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:53.058177 511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:53.058348 511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:53.058460 511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 07 19:41:53 dist-test-slave-vq2q krb5kdc[32707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131713, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:53.079748 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.058541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:42141 (local address 127.30.208.62:35625)
0807 19:41:53.058738 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:53.058745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:53.060104 (+ 1359us) server_negotiation.cc:407] Connection header received
0807 19:41:53.062216 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:53.062222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:53.062319 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:53.062459 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:53.064728 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.065789 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.068278 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.068843 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.068962 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.073857 (+ 4895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:53.073863 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:53.073868 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:53.073910 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:53.076636 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:53.077424 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:53.077428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:53.077430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:53.077568 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:53.077986 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:53.077990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:53.077992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:53.078293 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:53.078424 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:53.079169 (+ 745us) server_negotiation.cc:299] Negotiation successful
0807 19:41:53.079399 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:41:53.081041 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.059004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:35625 (local address 127.30.208.2:42141)
0807 19:41:53.059881 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:53.059941 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:53.061899 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:53.062821 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:53.062832 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:53.063584 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:53.064462 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.064495 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.065945 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.065951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.068076 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.068093 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.069093 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.069101 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.069409 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.070199 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:53.070239 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:53.073532 (+ 3293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:53.076783 (+ 3251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:53.076792 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:53.076809 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:53.077304 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:53.077675 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:53.077680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:53.077684 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:53.077870 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:53.078419 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:53.078427 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:53.078796 (+ 369us) client_negotiation.cc:769] Sending connection context
0807 19:41:53.079029 (+ 233us) client_negotiation.cc:240] Negotiation successful
0807 19:41:53.079564 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":383,"threads_started":1}
I20260807 19:41:53.103544 511 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:33199
I20260807 19:41:53.103564 653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:33199 every 8 connection(s)
I20260807 19:41:53.104211 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.087966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:46693 (local address 127.30.208.62:35625)
0807 19:41:53.088137 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:53.088143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:53.089108 (+ 965us) server_negotiation.cc:407] Connection header received
0807 19:41:53.089271 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:53.089277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:53.089361 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:53.089463 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:53.090571 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.091774 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.093916 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.094610 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.094748 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.097503 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:53.097510 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:53.097515 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:53.097560 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:53.101072 (+ 3512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:53.101985 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:53.101989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:53.101991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:53.102123 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:53.102565 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:53.102570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:53.102573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:53.103058 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:53.103214 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:53.103735 (+ 521us) server_negotiation.cc:299] Negotiation successful
0807 19:41:53.103984 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 19:41:53.104508 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.088183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:35625 (local address 127.30.208.2:46693)
0807 19:41:53.089002 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:53.089021 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:53.089139 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:53.089521 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:53.089527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:53.090094 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:53.090431 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.090442 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.091946 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.091952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.093753 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.093771 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.095018 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.095029 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.095209 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.095880 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:53.095906 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:53.097278 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:53.101257 (+ 3979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:53.101262 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:53.101271 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:53.101853 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:53.102257 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:53.102266 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:53.102271 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:53.102436 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:53.103203 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:53.103207 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:53.103445 (+ 238us) client_negotiation.cc:769] Sending connection context
0807 19:41:53.103599 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 19:41:53.103956 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":534,"threads_started":1}
I20260807 19:41:53.105266 511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:41:53.107275 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 511
I20260807 19:41:53.107496 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:41:53.107600 654 heartbeater.cc:347] Connected to a master server at 127.30.208.62:35625
I20260807 19:41:53.107808 654 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:53.108009 31552 external_mini_cluster.cc:1428] Setting key e32341e8ddcb6cb164628584afcf5772
I20260807 19:41:53.108566 654 heartbeater.cc:511] Master 127.30.208.62:35625 requested a full tablet report, sending...
I20260807 19:41:53.109947 32754 ts_manager.cc:194] Registered new tserver with Master: fa290f6d93644347af0c41cf053b09d3 (127.30.208.2:33199)
I20260807 19:41:53.110725 32754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:42141
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:41:53.154162 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:35625
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:35625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:53.482700 659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:53.483033 659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:53.483178 659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:53.490530 659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:53.490643 659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:53.490734 659 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:41:53.490816 659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:41:53.500825 659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:35625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.208.62:35625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:53.502141 659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:53.503434 659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:53.516512 665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:53.516516 664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:53.516520 667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:53.518491 659 server_base.cc:1034] running on GCE node
I20260807 19:41:53.519068 659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:53.519965 659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:53.521203 659 hybrid_clock.cc:648] HybridClock initialized: now 1786131713521110 us; error 49 us; skew 500 ppm
I20260807 19:41:53.521510 659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:53 dist-test-slave-vq2q krb5kdc[32707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131713, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:53.524804 659 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:41:53.526427 659 webserver.cc:460] Webserver started at http://127.30.208.3:40097/ using document root <none> and password file <none>
I20260807 19:41:53.527150 659 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:53.527253 659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:53.527587 659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:53.530611 659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "532f5ac5de224f6a9623f409878be921"
format_stamp: "Formatted at 2026-08-07 19:41:53 on dist-test-slave-vq2q"
server_key: "1fbf89b01b0763d73d333f20cfdab0b8"
server_key_iv: "2ae4c28edb744fb93950b3d7932a1d10"
server_key_version: "encryptionkey@0"
I20260807 19:41:53.531402 659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "532f5ac5de224f6a9623f409878be921"
format_stamp: "Formatted at 2026-08-07 19:41:53 on dist-test-slave-vq2q"
server_key: "1fbf89b01b0763d73d333f20cfdab0b8"
server_key_iv: "2ae4c28edb744fb93950b3d7932a1d10"
server_key_version: "encryptionkey@0"
I20260807 19:41:53.536502 659 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:41:53.540069 674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:53.541431 659 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:41:53.541589 659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "532f5ac5de224f6a9623f409878be921"
format_stamp: "Formatted at 2026-08-07 19:41:53 on dist-test-slave-vq2q"
server_key: "1fbf89b01b0763d73d333f20cfdab0b8"
server_key_iv: "2ae4c28edb744fb93950b3d7932a1d10"
server_key_version: "encryptionkey@0"
I20260807 19:41:53.541849 659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:53.574972 659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:53.579531 659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:53.579797 659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:53.582827 659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:53.582948 659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:53.583160 659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:53.583237 659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:53.598039 505 heartbeater.cc:503] Master 127.30.208.62:35625 was elected leader, sending a full tablet report...
Aug 07 19:41:53 dist-test-slave-vq2q krb5kdc[32707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131713, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:53.606523 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.583163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:48181 (local address 127.30.208.62:35625)
0807 19:41:53.583377 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:53.583383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:53.584925 (+ 1542us) server_negotiation.cc:407] Connection header received
0807 19:41:53.587268 (+ 2343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:53.587274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:53.587362 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:53.587466 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:53.589962 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.591103 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.593970 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.594635 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.594774 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.600309 (+ 5535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:53.600314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:53.600317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:53.600350 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:53.602895 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:53.603633 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:53.603638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:53.603640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:53.603774 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:53.604187 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:53.604191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:53.604193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:53.604480 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:53.604611 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:53.605791 (+ 1180us) server_negotiation.cc:299] Negotiation successful
0807 19:41:53.606074 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:41:53.607044 684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.583864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:35625 (local address 127.30.208.3:48181)
0807 19:41:53.584725 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:53.584781 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:53.586953 (+ 2172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:53.587898 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:53.587908 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:53.588763 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:53.589700 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.589744 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.591354 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.591363 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.593788 (+ 2425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.593809 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.594951 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.594960 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.595322 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.596400 (+ 1078us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:53.596443 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:53.600043 (+ 3600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:53.603047 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:53.603058 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:53.603075 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:53.603508 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:53.603880 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:53.603885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:53.603889 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:53.604072 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:53.604605 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:53.604612 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:53.604993 (+ 381us) client_negotiation.cc:769] Sending connection context
0807 19:41:53.605269 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 19:41:53.605846 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":395,"threads_started":1}
I20260807 19:41:53.628746 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.613666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:36941 (local address 127.30.208.62:35625)
0807 19:41:53.613838 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:53.613844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:53.614881 (+ 1037us) server_negotiation.cc:407] Connection header received
0807 19:41:53.615094 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:53.615100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:53.615196 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:53.615301 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:53.616712 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.617474 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.619428 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.620105 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.620251 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.622881 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:53.622888 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:53.622892 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:53.622934 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:53.626029 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:53.626911 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:53.626914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:53.626917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:53.627043 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:53.627408 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:53.627411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:53.627413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:53.627668 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:53.627808 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:53.628310 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 19:41:53.628487 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:41:53.629074 738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.613785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:35625 (local address 127.30.208.3:36941)
0807 19:41:53.614751 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:53.614773 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:53.614946 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:53.615351 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:53.615356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:53.616052 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:53.616568 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.616580 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.617619 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.617625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.619261 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.619281 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.620342 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.620350 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.620505 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.621229 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:53.621259 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:53.622663 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:53.626166 (+ 3503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:53.626172 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:53.626177 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:53.626801 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:53.627136 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:53.627141 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:53.627145 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:53.627308 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:53.627769 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:53.627774 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:53.628024 (+ 250us) client_negotiation.cc:769] Sending connection context
0807 19:41:53.628207 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 19:41:53.628568 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"thread_start_us":626,"threads_started":1}
I20260807 19:41:53.636775 659 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:35895
I20260807 19:41:53.636857 802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:35895 every 8 connection(s)
I20260807 19:41:53.638147 659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:41:53.640441 803 heartbeater.cc:347] Connected to a master server at 127.30.208.62:35625
I20260807 19:41:53.640727 803 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:53.641443 803 heartbeater.cc:511] Master 127.30.208.62:35625 requested a full tablet report, sending...
I20260807 19:41:53.642798 32754 ts_manager.cc:194] Registered new tserver with Master: 532f5ac5de224f6a9623f409878be921 (127.30.208.3:35895)
I20260807 19:41:53.643659 32754 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:48181
I20260807 19:41:53.643996 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 659
I20260807 19:41:53.644215 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:41:53.644737 31552 external_mini_cluster.cc:1428] Setting key 3595a39a312d49fd1719150ae5f09a92
I20260807 19:41:53.647776 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:41:53 dist-test-slave-vq2q krb5kdc[32707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131713, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:41:53 dist-test-slave-vq2q krb5kdc[32707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131713, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:53.697423 387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.683227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58102 (local address 127.30.208.62:35625)
0807 19:41:53.683405 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:53.683410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:53.683757 (+ 347us) server_negotiation.cc:407] Connection header received
0807 19:41:53.684028 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:53.684033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:53.684104 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:53.684219 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:53.685719 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.686504 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.687778 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.688251 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.688353 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.692203 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:53.692209 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:53.692212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:53.692263 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:53.694855 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:53.695467 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:53.695472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:53.695474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:53.695599 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:53.696112 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:53.696116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:53.696118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:53.696419 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:53.696626 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:53.696907 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 19:41:53.697085 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:41:53.701821 32754 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:41:53.708793 32754 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:41:53.732280 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.721100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:33199 (local address 127.0.0.1:44212)
0807 19:41:53.721963 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:53.722015 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:53.722204 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:53.723279 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:53.723289 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:53.723324 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:53.723794 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.723809 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.725993 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.726001 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.728677 (+ 2676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.728699 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.731137 (+ 2438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.731145 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.731396 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.731452 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 19:41:53.731597 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 19:41:53.731837 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":415,"threads_started":1}
I20260807 19:41:53.733124 819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.721378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44212 (local address 127.30.208.2:33199)
0807 19:41:53.721950 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:53.721959 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:53.722427 (+ 468us) server_negotiation.cc:407] Connection header received
0807 19:41:53.722548 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:53.722557 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:53.722808 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:53.723009 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:53.724045 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.725775 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.728922 (+ 3147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.730543 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.730701 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.730961 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:53.732563 (+ 1602us) server_negotiation.cc:299] Negotiation successful
0807 19:41:53.732782 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":1447,"threads_started":1}
I20260807 19:41:53.737838 581 tablet_service.cc:1467] Processing CreateTablet for tablet 071547a917a14cb8865ee954ba951313 (DEFAULT_TABLE table=test-table [id=11f697c0eb1f4d788898ddd42d272f3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:53.739394 581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 071547a917a14cb8865ee954ba951313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:53.742136 820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.721825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:35895 (local address 127.0.0.1:49892)
0807 19:41:53.722481 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:53.722498 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:53.722597 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:53.735310 (+ 12713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:53.735315 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:53.735343 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:53.735668 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.735678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.737495 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.737501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.739365 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.739381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.741344 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.741361 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.741538 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.741577 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 19:41:53.741649 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 19:41:53.741789 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":1208,"threads_started":1}
I20260807 19:41:53.743369 821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.722437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:42339 (local address 127.0.0.1:35332)
0807 19:41:53.733343 (+ 10906us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:53.733364 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:53.733501 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:53.736189 (+ 2688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:53.736194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:53.736223 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:53.736590 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.736604 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.738463 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.738469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.740680 (+ 2211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.740698 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.742597 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.742604 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.742773 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.742807 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:41:53.742895 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 19:41:53.743029 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10751,"thread_start_us":11136,"threads_started":1}
I20260807 19:41:53.743388 823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.722981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35332 (local address 127.30.208.1:42339)
0807 19:41:53.735623 (+ 12642us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:53.735633 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:53.735661 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:41:53.735758 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:53.735765 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:53.735985 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:53.736194 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:53.736777 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.738325 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.740910 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.742404 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.742538 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.742757 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:53.742953 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 19:41:53.743115 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12482,"thread_start_us":12314,"threads_started":1}
I20260807 19:41:53.743389 822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.722576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49892 (local address 127.30.208.3:35895)
0807 19:41:53.734228 (+ 11652us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:53.734238 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:53.734280 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:41:53.734393 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:53.734403 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:53.734698 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:53.734930 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:53.735896 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.737324 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.739575 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.741086 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.742548 (+ 1462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.742800 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:53.742878 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 19:41:53.743051 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11435,"thread_start_us":11222,"threads_started":1}
I20260807 19:41:53.747061 728 tablet_service.cc:1467] Processing CreateTablet for tablet 071547a917a14cb8865ee954ba951313 (DEFAULT_TABLE table=test-table [id=11f697c0eb1f4d788898ddd42d272f3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:53.747089 431 tablet_service.cc:1467] Processing CreateTablet for tablet 071547a917a14cb8865ee954ba951313 (DEFAULT_TABLE table=test-table [id=11f697c0eb1f4d788898ddd42d272f3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:53.748377 728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 071547a917a14cb8865ee954ba951313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:53.748487 431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 071547a917a14cb8865ee954ba951313. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:53.757547 824 tablet_bootstrap.cc:492] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3: Bootstrap starting.
I20260807 19:41:53.760684 824 tablet_bootstrap.cc:654] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:53.762033 824 log.cc:824] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:53.762701 825 tablet_bootstrap.cc:492] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921: Bootstrap starting.
I20260807 19:41:53.763315 826 tablet_bootstrap.cc:492] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087: Bootstrap starting.
I20260807 19:41:53.766392 825 tablet_bootstrap.cc:654] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:53.766770 826 tablet_bootstrap.cc:654] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:53.766981 824 tablet_bootstrap.cc:492] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3: No bootstrap required, opened a new log
I20260807 19:41:53.767269 824 ts_tablet_manager.cc:1397] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 19:41:53.767797 825 log.cc:824] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:53.768005 826 log.cc:824] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:53.770660 826 tablet_bootstrap.cc:492] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087: No bootstrap required, opened a new log
I20260807 19:41:53.770803 825 tablet_bootstrap.cc:492] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921: No bootstrap required, opened a new log
I20260807 19:41:53.770936 826 ts_tablet_manager.cc:1397] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:41:53.771072 825 ts_tablet_manager.cc:1397] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:41:53.777698 824 raft_consensus.cc:348] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.778153 824 raft_consensus.cc:374] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:53.778244 824 raft_consensus.cc:729] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa290f6d93644347af0c41cf053b09d3, State: Initialized, Role: FOLLOWER
I20260807 19:41:53.778946 824 consensus_queue.cc:260] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.780807 826 raft_consensus.cc:348] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.781250 826 raft_consensus.cc:374] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:53.781349 826 raft_consensus.cc:729] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca85bc96b104c22a05f824fdfcae087, State: Initialized, Role: FOLLOWER
I20260807 19:41:53.781456 825 raft_consensus.cc:348] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.781874 825 raft_consensus.cc:374] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:53.781956 825 raft_consensus.cc:729] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 532f5ac5de224f6a9623f409878be921, State: Initialized, Role: FOLLOWER
I20260807 19:41:53.782099 826 consensus_queue.cc:260] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.782652 825 consensus_queue.cc:260] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.786248 654 heartbeater.cc:503] Master 127.30.208.62:35625 was elected leader, sending a full tablet report...
I20260807 19:41:53.787320 824 ts_tablet_manager.cc:1428] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3: Time spent starting tablet: real 0.020s user 0.013s sys 0.007s
I20260807 19:41:53.788971 826 ts_tablet_manager.cc:1428] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087: Time spent starting tablet: real 0.018s user 0.006s sys 0.012s
I20260807 19:41:53.790061 803 heartbeater.cc:503] Master 127.30.208.62:35625 was elected leader, sending a full tablet report...
I20260807 19:41:53.790505 825 ts_tablet_manager.cc:1428] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921: Time spent starting tablet: real 0.019s user 0.013s sys 0.008s
I20260807 19:41:53.797078 830 raft_consensus.cc:482] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:41:53.797405 830 raft_consensus.cc:504] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.799154 830 leader_election.cc:288] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca85bc96b104c22a05f824fdfcae087 (127.30.208.1:42339), 532f5ac5de224f6a9623f409878be921 (127.30.208.3:35895)
I20260807 19:41:53.806726 822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.799487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:42701 (local address 127.30.208.3:35895)
0807 19:41:53.799692 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:53.799698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:53.800196 (+ 498us) server_negotiation.cc:407] Connection header received
0807 19:41:53.800398 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:53.800402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:53.800460 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:53.800531 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:53.801024 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.802041 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.804532 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.805783 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.805958 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.806038 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:53.806369 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 19:41:53.806480 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 19:41:53.806915 833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.799402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:35895 (local address 127.30.208.2:42701)
0807 19:41:53.800096 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:53.800118 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:53.800275 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:53.800552 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:53.800556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:53.800579 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:53.800879 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.800890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.802163 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.802166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.804354 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.804370 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.806025 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.806030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.806193 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.806233 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 19:41:53.806341 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 19:41:53.806485 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":30,"thread_start_us":318,"threads_started":1}
I20260807 19:41:53.806967 823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.799631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:53809 (local address 127.30.208.1:42339)
0807 19:41:53.799821 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:53.799827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:53.800441 (+ 614us) server_negotiation.cc:407] Connection header received
0807 19:41:53.800627 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:53.800632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:53.800762 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:53.800837 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:53.801515 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.802448 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.804750 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.806036 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.806176 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.806227 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:53.806596 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 19:41:53.806728 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:41:53.807641 834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.799401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:42339 (local address 127.30.208.2:53809)
0807 19:41:53.800360 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:53.800376 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:53.800484 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:53.800890 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:53.800895 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:53.800945 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:53.801355 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.801366 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.802582 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.802587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.804598 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:53.804609 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.806245 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.806251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:53.806442 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.806473 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 19:41:53.806556 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 19:41:53.806678 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"mutex_wait_us":327,"thread_start_us":296,"threads_started":1}
I20260807 19:41:53.807927 755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "071547a917a14cb8865ee954ba951313" candidate_uuid: "fa290f6d93644347af0c41cf053b09d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "532f5ac5de224f6a9623f409878be921" is_pre_election: true
I20260807 19:41:53.808439 755 raft_consensus.cc:2454] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa290f6d93644347af0c41cf053b09d3 in term 0.
I20260807 19:41:53.808511 451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "071547a917a14cb8865ee954ba951313" candidate_uuid: "fa290f6d93644347af0c41cf053b09d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca85bc96b104c22a05f824fdfcae087" is_pre_election: true
I20260807 19:41:53.809023 451 raft_consensus.cc:2454] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa290f6d93644347af0c41cf053b09d3 in term 0.
I20260807 19:41:53.809068 531 leader_election.cc:302] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 532f5ac5de224f6a9623f409878be921, fa290f6d93644347af0c41cf053b09d3; no voters:
I20260807 19:41:53.809567 830 raft_consensus.cc:2781] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:41:53.809736 830 raft_consensus.cc:482] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:41:53.809847 830 raft_consensus.cc:3037] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:53.812923 830 raft_consensus.cc:504] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.813732 830 leader_election.cc:288] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [CANDIDATE]: Term 1 election: Requested vote from peers bca85bc96b104c22a05f824fdfcae087 (127.30.208.1:42339), 532f5ac5de224f6a9623f409878be921 (127.30.208.3:35895)
I20260807 19:41:53.814270 755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "071547a917a14cb8865ee954ba951313" candidate_uuid: "fa290f6d93644347af0c41cf053b09d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "532f5ac5de224f6a9623f409878be921"
I20260807 19:41:53.814364 451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "071547a917a14cb8865ee954ba951313" candidate_uuid: "fa290f6d93644347af0c41cf053b09d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca85bc96b104c22a05f824fdfcae087"
I20260807 19:41:53.814507 755 raft_consensus.cc:3037] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:53.814567 451 raft_consensus.cc:3037] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:53.817807 755 raft_consensus.cc:2454] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa290f6d93644347af0c41cf053b09d3 in term 1.
I20260807 19:41:53.817807 451 raft_consensus.cc:2454] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa290f6d93644347af0c41cf053b09d3 in term 1.
I20260807 19:41:53.818271 531 leader_election.cc:302] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 532f5ac5de224f6a9623f409878be921, fa290f6d93644347af0c41cf053b09d3; no voters:
I20260807 19:41:53.818609 830 raft_consensus.cc:2781] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:41:53.819072 830 raft_consensus.cc:686] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [term 1 LEADER]: Becoming Leader. State: Replica: fa290f6d93644347af0c41cf053b09d3, State: Running, Role: LEADER
I20260807 19:41:53.819626 830 consensus_queue.cc:237] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.824343 32753 catalog_manager.cc:5495] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to fa290f6d93644347af0c41cf053b09d3 (127.30.208.2). New cstate: current_term: 1 leader_uuid: "fa290f6d93644347af0c41cf053b09d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } health_report { overall_health: UNKNOWN } } }
W20260807 19:41:53.840904 507 tablet.cc:2369] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:41:53.852002 819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:53.846535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44224 (local address 127.30.208.2:33199)
0807 19:41:53.846749 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:53.846754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:53.846773 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:41:53.847025 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:53.847029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:53.847107 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:53.847223 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:41:53.847576 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.848297 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.849702 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:53.850163 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:53.850259 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:53.850892 (+ 633us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:41:53.851431 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:41:53.851537 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:53.851673 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 19:41:53.851744 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 19:41:53.857533 655 tablet.cc:2369] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:41:53.860175 451 raft_consensus.cc:1261] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087 [term 1 FOLLOWER]: Refusing update from remote peer fa290f6d93644347af0c41cf053b09d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:41:53.860188 755 raft_consensus.cc:1261] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921 [term 1 FOLLOWER]: Refusing update from remote peer fa290f6d93644347af0c41cf053b09d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:41:53.861070 835 consensus_queue.cc:1035] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:53.861497 830 consensus_queue.cc:1035] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:53.874526 840 mvcc.cc:205] Tried to move back new op lower bound from 7315995499961241600 to 7315995499809304576. Current Snapshot: MvccSnapshot[applied={T|T < 7315995499961241600}]
I20260807 19:41:53.891999 32753 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:41:53.892323 32753 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:41:53.896387 32753 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749: Sending DeleteTablet for 3 replicas of tablet 071547a917a14cb8865ee954ba951313
I20260807 19:41:53.897608 581 tablet_service.cc:1514] Processing DeleteTablet for tablet 071547a917a14cb8865ee954ba951313 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:53 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:44212
I20260807 19:41:53.897594 728 tablet_service.cc:1514] Processing DeleteTablet for tablet 071547a917a14cb8865ee954ba951313 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:53 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:49892
I20260807 19:41:53.897702 431 tablet_service.cc:1514] Processing DeleteTablet for tablet 071547a917a14cb8865ee954ba951313 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:53 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:35332
I20260807 19:41:53.905612 32753 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:41:53.906404 32753 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:41:53.909931 851 tablet_replica.cc:330] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087: stopping tablet replica
I20260807 19:41:53.910463 851 raft_consensus.cc:2229] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:41:53.910547 850 tablet_replica.cc:330] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921: stopping tablet replica
I20260807 19:41:53.911135 850 raft_consensus.cc:2229] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:41:53.911690 850 raft_consensus.cc:2258] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:53.912562 849 tablet_replica.cc:330] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3: stopping tablet replica
I20260807 19:41:53.913018 849 raft_consensus.cc:2229] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:41:53.913775 849 raft_consensus.cc:2258] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:53.914119 850 ts_tablet_manager.cc:1905] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:53.914299 851 raft_consensus.cc:2258] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:53.915176 431 tablet_service.cc:1467] Processing CreateTablet for tablet bda7757230a34d05a6bcf8fd4b4cfe63 (DEFAULT_TABLE table=test-table [id=c56f78b354ba487786c741491ecf1634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:53.915319 581 tablet_service.cc:1467] Processing CreateTablet for tablet bda7757230a34d05a6bcf8fd4b4cfe63 (DEFAULT_TABLE table=test-table [id=c56f78b354ba487786c741491ecf1634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:53.915642 431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bda7757230a34d05a6bcf8fd4b4cfe63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:53.915956 581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bda7757230a34d05a6bcf8fd4b4cfe63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:53.921051 850 ts_tablet_manager.cc:1918] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:41:53.921253 850 log.cc:1196] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/071547a917a14cb8865ee954ba951313
I20260807 19:41:53.921805 850 ts_tablet_manager.cc:1939] T 071547a917a14cb8865ee954ba951313 P 532f5ac5de224f6a9623f409878be921: Deleting consensus metadata
I20260807 19:41:53.922370 727 tablet_service.cc:1467] Processing CreateTablet for tablet bda7757230a34d05a6bcf8fd4b4cfe63 (DEFAULT_TABLE table=test-table [id=c56f78b354ba487786c741491ecf1634]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:53.922869 727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bda7757230a34d05a6bcf8fd4b4cfe63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:53.923834 826 tablet_bootstrap.cc:492] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087: Bootstrap starting.
I20260807 19:41:53.925103 851 ts_tablet_manager.cc:1905] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:53.925539 826 tablet_bootstrap.cc:654] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:53.925614 849 ts_tablet_manager.cc:1905] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:53.929922 825 tablet_bootstrap.cc:492] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921: Bootstrap starting.
I20260807 19:41:53.931566 825 tablet_bootstrap.cc:654] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:53.931625 32741 catalog_manager.cc:4841] TS 532f5ac5de224f6a9623f409878be921 (127.30.208.3:35895): tablet 071547a917a14cb8865ee954ba951313 (table test-table [id=11f697c0eb1f4d788898ddd42d272f3b]) successfully deleted
I20260807 19:41:53.932440 826 tablet_bootstrap.cc:492] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087: No bootstrap required, opened a new log
I20260807 19:41:53.932608 826 ts_tablet_manager.cc:1397] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:41:53.933704 826 raft_consensus.cc:348] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.933880 849 ts_tablet_manager.cc:1918] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:41:53.933996 826 raft_consensus.cc:374] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:53.934041 849 log.cc:1196] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/071547a917a14cb8865ee954ba951313
I20260807 19:41:53.934093 826 raft_consensus.cc:729] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca85bc96b104c22a05f824fdfcae087, State: Initialized, Role: FOLLOWER
I20260807 19:41:53.934568 849 ts_tablet_manager.cc:1939] T 071547a917a14cb8865ee954ba951313 P fa290f6d93644347af0c41cf053b09d3: Deleting consensus metadata
I20260807 19:41:53.934511 826 consensus_queue.cc:260] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.935261 826 ts_tablet_manager.cc:1428] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 19:41:53.936929 825 tablet_bootstrap.cc:492] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921: No bootstrap required, opened a new log
I20260807 19:41:53.936986 824 tablet_bootstrap.cc:492] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3: Bootstrap starting.
I20260807 19:41:53.937103 825 ts_tablet_manager.cc:1397] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20260807 19:41:53.937448 851 ts_tablet_manager.cc:1918] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:41:53.937671 851 log.cc:1196] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/071547a917a14cb8865ee954ba951313
I20260807 19:41:53.938016 825 raft_consensus.cc:348] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.938258 825 raft_consensus.cc:374] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:53.938381 825 raft_consensus.cc:729] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 532f5ac5de224f6a9623f409878be921, State: Initialized, Role: FOLLOWER
I20260807 19:41:53.938668 32738 catalog_manager.cc:4841] TS fa290f6d93644347af0c41cf053b09d3 (127.30.208.2:33199): tablet 071547a917a14cb8865ee954ba951313 (table test-table [id=11f697c0eb1f4d788898ddd42d272f3b]) successfully deleted
I20260807 19:41:53.938913 824 tablet_bootstrap.cc:654] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:53.938742 825 consensus_queue.cc:260] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.939551 825 ts_tablet_manager.cc:1428] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:41:53.942538 851 ts_tablet_manager.cc:1939] T 071547a917a14cb8865ee954ba951313 P bca85bc96b104c22a05f824fdfcae087: Deleting consensus metadata
I20260807 19:41:53.943161 824 tablet_bootstrap.cc:492] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3: No bootstrap required, opened a new log
I20260807 19:41:53.943359 824 ts_tablet_manager.cc:1397] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.003s
I20260807 19:41:53.944311 824 raft_consensus.cc:348] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.944615 824 raft_consensus.cc:374] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:53.944753 824 raft_consensus.cc:729] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa290f6d93644347af0c41cf053b09d3, State: Initialized, Role: FOLLOWER
I20260807 19:41:53.945057 824 consensus_queue.cc:260] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:53.946599 824 ts_tablet_manager.cc:1428] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 19:41:53.947459 32738 catalog_manager.cc:4841] TS bca85bc96b104c22a05f824fdfcae087 (127.30.208.1:42339): tablet 071547a917a14cb8865ee954ba951313 (table test-table [id=11f697c0eb1f4d788898ddd42d272f3b]) successfully deleted
W20260807 19:41:54.140944 804 tablet.cc:2369] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:41:54.312239 832 raft_consensus.cc:482] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:41:54.312472 832 raft_consensus.cc:504] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:54.314018 832 leader_election.cc:288] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca85bc96b104c22a05f824fdfcae087 (127.30.208.1:42339), fa290f6d93644347af0c41cf053b09d3 (127.30.208.2:33199)
I20260807 19:41:54.321002 855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.314208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:42339 (local address 127.30.208.3:42029)
0807 19:41:54.314680 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:54.314700 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:54.314834 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:54.315919 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:54.315925 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:54.315953 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:54.316144 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:54.316153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.317675 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.317680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:54.319083 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:54.319095 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.320215 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.320220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:54.320394 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.320428 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:41:54.320516 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 19:41:54.320647 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":38,"thread_start_us":227,"threads_started":1}
I20260807 19:41:54.321043 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.314387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:42029 (local address 127.30.208.1:42339)
0807 19:41:54.315599 (+ 1212us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:54.315605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:54.315626 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:41:54.315700 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:54.315705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:54.315784 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:54.315881 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:54.316314 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.317513 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.319241 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.320060 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.320251 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.320329 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:54.320552 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 19:41:54.320697 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1086,"thread_start_us":1018,"threads_started":1}
I20260807 19:41:54.321940 451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bda7757230a34d05a6bcf8fd4b4cfe63" candidate_uuid: "532f5ac5de224f6a9623f409878be921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca85bc96b104c22a05f824fdfcae087" is_pre_election: true
I20260807 19:41:54.322198 451 raft_consensus.cc:2454] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 532f5ac5de224f6a9623f409878be921 in term 0.
I20260807 19:41:54.322436 857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.314885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:33199 (local address 127.30.208.3:37459)
0807 19:41:54.315908 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:54.315927 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:54.316118 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:54.316444 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:54.316448 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:54.316471 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:54.316688 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:54.316705 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.317695 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.317700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:54.319782 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:54.319796 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.321657 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.321663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:54.321851 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.321875 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:41:54.321956 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 19:41:54.322100 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":902,"mutex_wait_us":4,"thread_start_us":684,"threads_started":1}
I20260807 19:41:54.322745 675 leader_election.cc:302] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 532f5ac5de224f6a9623f409878be921, bca85bc96b104c22a05f824fdfcae087; no voters:
I20260807 19:41:54.323107 832 raft_consensus.cc:2781] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:41:54.323201 819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.315181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:37459 (local address 127.30.208.2:33199)
0807 19:41:54.315391 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:54.315398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:54.316009 (+ 611us) server_negotiation.cc:407] Connection header received
0807 19:41:54.316273 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:54.316279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:54.316342 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:54.316416 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:54.316834 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.317566 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.319949 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.321116 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.321488 (+ 372us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.321560 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:54.322003 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 19:41:54.322135 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:41:54.323230 832 raft_consensus.cc:482] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:41:54.323333 832 raft_consensus.cc:3037] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:54.323953 605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bda7757230a34d05a6bcf8fd4b4cfe63" candidate_uuid: "532f5ac5de224f6a9623f409878be921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa290f6d93644347af0c41cf053b09d3" is_pre_election: true
I20260807 19:41:54.324311 605 raft_consensus.cc:2454] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 532f5ac5de224f6a9623f409878be921 in term 0.
I20260807 19:41:54.325829 832 raft_consensus.cc:504] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:54.326424 832 leader_election.cc:288] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [CANDIDATE]: Term 1 election: Requested vote from peers bca85bc96b104c22a05f824fdfcae087 (127.30.208.1:42339), fa290f6d93644347af0c41cf053b09d3 (127.30.208.2:33199)
I20260807 19:41:54.326984 605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bda7757230a34d05a6bcf8fd4b4cfe63" candidate_uuid: "532f5ac5de224f6a9623f409878be921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa290f6d93644347af0c41cf053b09d3"
I20260807 19:41:54.327025 451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bda7757230a34d05a6bcf8fd4b4cfe63" candidate_uuid: "532f5ac5de224f6a9623f409878be921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca85bc96b104c22a05f824fdfcae087"
I20260807 19:41:54.327178 605 raft_consensus.cc:3037] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:54.327255 451 raft_consensus.cc:3037] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:54.329425 605 raft_consensus.cc:2454] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 532f5ac5de224f6a9623f409878be921 in term 1.
I20260807 19:41:54.329614 451 raft_consensus.cc:2454] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 532f5ac5de224f6a9623f409878be921 in term 1.
I20260807 19:41:54.329833 675 leader_election.cc:302] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 532f5ac5de224f6a9623f409878be921, fa290f6d93644347af0c41cf053b09d3; no voters:
I20260807 19:41:54.330108 832 raft_consensus.cc:2781] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:41:54.330528 832 raft_consensus.cc:686] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [term 1 LEADER]: Becoming Leader. State: Replica: 532f5ac5de224f6a9623f409878be921, State: Running, Role: LEADER
I20260807 19:41:54.330928 832 consensus_queue.cc:237] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:54.334069 32754 catalog_manager.cc:5495] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 reported cstate change: term changed from 0 to 1, leader changed from <none> to 532f5ac5de224f6a9623f409878be921 (127.30.208.3). New cstate: current_term: 1 leader_uuid: "532f5ac5de224f6a9623f409878be921" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } health_report { overall_health: HEALTHY } } }
Aug 07 19:41:54 dist-test-slave-vq2q krb5kdc[32707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131711, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:54.441363 865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.427682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:35625 (local address 127.0.0.1:58116)
0807 19:41:54.428674 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:54.428691 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:54.428874 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:54.429239 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:54.429243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:54.429953 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:54.430166 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:54.430175 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.431236 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.431241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:54.432253 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:54.432265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.432970 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.432976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:54.433116 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.433765 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:54.433814 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:54.436093 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:54.438930 (+ 2837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:54.438942 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:54.438948 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:54.439418 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:54.439800 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:54.439805 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:54.439808 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:54.440009 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:54.440504 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:54.440510 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:54.440676 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 19:41:54.440822 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 19:41:54.441031 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"thread_start_us":371,"threads_started":1}
I20260807 19:41:54.441536 866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.427765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58116 (local address 127.30.208.62:35625)
0807 19:41:54.428797 (+ 1032us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:54.428802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:54.428827 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:41:54.429011 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:54.429015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:54.429077 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:54.429171 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:54.430316 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.431109 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.432372 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.432795 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.432886 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.436267 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:54.436272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:54.436275 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:54.436303 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:54.438811 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:54.439548 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:54.439552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:54.439554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:54.439677 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:54.440105 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:54.440108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:54.440110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:54.440393 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:54.440521 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:54.440908 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 19:41:54.441158 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":866,"thread_start_us":709,"threads_started":1}
I20260807 19:41:54.446784 32754 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:58116:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 19:41:54.447834 32754 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:41:54.454452 431 tablet_service.cc:1467] Processing CreateTablet for tablet cc6c9d6cffef438e95dde8abd5c96771 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0384e53d915943a5836b403a2ef5b909]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:41:54.454453 581 tablet_service.cc:1467] Processing CreateTablet for tablet cc6c9d6cffef438e95dde8abd5c96771 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0384e53d915943a5836b403a2ef5b909]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:41:54.454599 727 tablet_service.cc:1467] Processing CreateTablet for tablet cc6c9d6cffef438e95dde8abd5c96771 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0384e53d915943a5836b403a2ef5b909]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:41:54.455005 581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc6c9d6cffef438e95dde8abd5c96771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:54.455005 431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc6c9d6cffef438e95dde8abd5c96771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:54.455061 727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc6c9d6cffef438e95dde8abd5c96771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:54.460654 867 tablet_bootstrap.cc:492] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921: Bootstrap starting.
I20260807 19:41:54.462069 867 tablet_bootstrap.cc:654] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:54.465771 870 tablet_bootstrap.cc:492] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087: Bootstrap starting.
I20260807 19:41:54.466300 869 tablet_bootstrap.cc:492] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3: Bootstrap starting.
I20260807 19:41:54.467136 867 tablet_bootstrap.cc:492] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921: No bootstrap required, opened a new log
I20260807 19:41:54.467307 867 ts_tablet_manager.cc:1397] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 19:41:54.467835 870 tablet_bootstrap.cc:654] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:54.467963 869 tablet_bootstrap.cc:654] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:54.468128 867 raft_consensus.cc:348] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:54.468369 867 raft_consensus.cc:374] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:54.468454 867 raft_consensus.cc:729] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 532f5ac5de224f6a9623f409878be921, State: Initialized, Role: FOLLOWER
I20260807 19:41:54.468741 867 consensus_queue.cc:260] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:54.469362 832 tablet_replica.cc:439] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } } }
I20260807 19:41:54.469699 832 tablet_replica.cc:442] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:41:54.470213 867 ts_tablet_manager.cc:1428] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 19:41:54.472692 870 tablet_bootstrap.cc:492] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087: No bootstrap required, opened a new log
I20260807 19:41:54.472862 870 ts_tablet_manager.cc:1397] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 19:41:54.473244 869 tablet_bootstrap.cc:492] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3: No bootstrap required, opened a new log
I20260807 19:41:54.473395 869 ts_tablet_manager.cc:1397] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20260807 19:41:54.473668 870 raft_consensus.cc:348] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:54.473870 870 raft_consensus.cc:374] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:54.473964 870 raft_consensus.cc:729] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bca85bc96b104c22a05f824fdfcae087, State: Initialized, Role: FOLLOWER
I20260807 19:41:54.474188 869 raft_consensus.cc:348] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:54.474242 870 consensus_queue.cc:260] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:54.474458 869 raft_consensus.cc:374] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:54.474552 869 raft_consensus.cc:729] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa290f6d93644347af0c41cf053b09d3, State: Initialized, Role: FOLLOWER
I20260807 19:41:54.474799 869 consensus_queue.cc:260] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:54.475346 870 ts_tablet_manager.cc:1428] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:41:54.475962 869 ts_tablet_manager.cc:1428] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 19:41:54.475919 875 tablet_replica.cc:439] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } } }
I20260807 19:41:54.476272 875 tablet_replica.cc:442] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:41:54.476639 874 tablet_replica.cc:439] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } } }
I20260807 19:41:54.476971 874 tablet_replica.cc:442] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:41:54.675489 832 raft_consensus.cc:482] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:41:54.675773 832 raft_consensus.cc:504] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:54.676476 832 leader_election.cc:288] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bca85bc96b104c22a05f824fdfcae087 (127.30.208.1:42339), fa290f6d93644347af0c41cf053b09d3 (127.30.208.2:33199)
I20260807 19:41:54.677208 451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc6c9d6cffef438e95dde8abd5c96771" candidate_uuid: "532f5ac5de224f6a9623f409878be921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca85bc96b104c22a05f824fdfcae087" is_pre_election: true
I20260807 19:41:54.677289 605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc6c9d6cffef438e95dde8abd5c96771" candidate_uuid: "532f5ac5de224f6a9623f409878be921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa290f6d93644347af0c41cf053b09d3" is_pre_election: true
I20260807 19:41:54.677500 451 raft_consensus.cc:2454] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 532f5ac5de224f6a9623f409878be921 in term 0.
I20260807 19:41:54.677579 605 raft_consensus.cc:2454] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 532f5ac5de224f6a9623f409878be921 in term 0.
I20260807 19:41:54.677963 675 leader_election.cc:302] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 532f5ac5de224f6a9623f409878be921, bca85bc96b104c22a05f824fdfcae087; no voters:
I20260807 19:41:54.678278 832 raft_consensus.cc:2781] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:41:54.678370 832 raft_consensus.cc:482] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:41:54.678493 832 raft_consensus.cc:3037] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:54.681313 832 raft_consensus.cc:504] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:54.681932 832 leader_election.cc:288] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [CANDIDATE]: Term 1 election: Requested vote from peers bca85bc96b104c22a05f824fdfcae087 (127.30.208.1:42339), fa290f6d93644347af0c41cf053b09d3 (127.30.208.2:33199)
I20260807 19:41:54.682528 451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc6c9d6cffef438e95dde8abd5c96771" candidate_uuid: "532f5ac5de224f6a9623f409878be921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bca85bc96b104c22a05f824fdfcae087"
I20260807 19:41:54.682590 605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc6c9d6cffef438e95dde8abd5c96771" candidate_uuid: "532f5ac5de224f6a9623f409878be921" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa290f6d93644347af0c41cf053b09d3"
I20260807 19:41:54.682819 451 raft_consensus.cc:3037] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:54.682864 605 raft_consensus.cc:3037] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:54.685068 605 raft_consensus.cc:2454] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 532f5ac5de224f6a9623f409878be921 in term 1.
I20260807 19:41:54.685101 451 raft_consensus.cc:2454] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 532f5ac5de224f6a9623f409878be921 in term 1.
I20260807 19:41:54.685580 675 leader_election.cc:302] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 532f5ac5de224f6a9623f409878be921, bca85bc96b104c22a05f824fdfcae087; no voters:
I20260807 19:41:54.685963 832 raft_consensus.cc:2781] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:41:54.686138 832 raft_consensus.cc:686] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [term 1 LEADER]: Becoming Leader. State: Replica: 532f5ac5de224f6a9623f409878be921, State: Running, Role: LEADER
I20260807 19:41:54.686409 832 consensus_queue.cc:237] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } }
I20260807 19:41:54.688025 858 tablet_replica.cc:439] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921: TxnStatusTablet state changed. Reason: New leader 532f5ac5de224f6a9623f409878be921. Latest consensus state: current_term: 1 leader_uuid: "532f5ac5de224f6a9623f409878be921" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } } }
I20260807 19:41:54.688361 858 tablet_replica.cc:442] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:41:54.689918 32753 catalog_manager.cc:5495] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 reported cstate change: term changed from 0 to 1, leader changed from <none> to 532f5ac5de224f6a9623f409878be921 (127.30.208.3). New cstate: current_term: 1 leader_uuid: "532f5ac5de224f6a9623f409878be921" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } health_report { overall_health: HEALTHY } } }
I20260807 19:41:54.692704 877 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 19:41:54.693043 877 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 19:41:54.719027 865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.711322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:35895 (local address 127.0.0.1:49898)
0807 19:41:54.711618 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:54.711643 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:54.711862 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:54.712890 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:54.712899 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:54.712922 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:41:54.713261 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:54.713278 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.714914 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.714920 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:54.715971 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:54.715985 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.717031 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.717036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:54.717257 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.717332 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:41:54.718537 (+ 1205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:41:54.718560 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:41:54.718624 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 19:41:54.718755 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 19:41:54.719342 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.711282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49898 (local address 127.30.208.3:35895)
0807 19:41:54.712060 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:54.712067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:54.712094 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:41:54.712211 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:54.712217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:54.712344 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:54.712636 (+ 292us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:41:54.713484 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.714739 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.716129 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.716855 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.716990 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.717577 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:41:54.718402 (+ 825us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:41:54.718561 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:54.718679 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 19:41:54.718771 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":442,"threads_started":1}
I20260807 19:41:54.724888 451 raft_consensus.cc:1261] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087 [term 1 FOLLOWER]: Refusing update from remote peer 532f5ac5de224f6a9623f409878be921: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:41:54.725307 605 raft_consensus.cc:1261] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3 [term 1 FOLLOWER]: Refusing update from remote peer 532f5ac5de224f6a9623f409878be921: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:41:54.725697 858 consensus_queue.cc:1035] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:54.726143 832 consensus_queue.cc:1035] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:54.735044 875 tablet_replica.cc:439] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3: TxnStatusTablet state changed. Reason: New leader 532f5ac5de224f6a9623f409878be921. Latest consensus state: current_term: 1 leader_uuid: "532f5ac5de224f6a9623f409878be921" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } } }
I20260807 19:41:54.735357 875 tablet_replica.cc:442] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:41:54.739574 874 tablet_replica.cc:439] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087: TxnStatusTablet state changed. Reason: New leader 532f5ac5de224f6a9623f409878be921. Latest consensus state: current_term: 1 leader_uuid: "532f5ac5de224f6a9623f409878be921" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } } }
I20260807 19:41:54.739892 874 tablet_replica.cc:442] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:41:54.739157 832 tablet_replica.cc:439] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "532f5ac5de224f6a9623f409878be921" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } } }
I20260807 19:41:54.740936 832 tablet_replica.cc:442] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:41:54.741601 875 tablet_replica.cc:439] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "532f5ac5de224f6a9623f409878be921" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } } }
I20260807 19:41:54.741866 875 tablet_replica.cc:442] T cc6c9d6cffef438e95dde8abd5c96771 P fa290f6d93644347af0c41cf053b09d3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:41:54.747890 832 tablet_replica.cc:439] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "532f5ac5de224f6a9623f409878be921" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } } }
I20260807 19:41:54.748209 832 tablet_replica.cc:442] T cc6c9d6cffef438e95dde8abd5c96771 P 532f5ac5de224f6a9623f409878be921: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:41:54.750926 874 tablet_replica.cc:439] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "532f5ac5de224f6a9623f409878be921" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 } } peers { permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 } } peers { permanent_uuid: "532f5ac5de224f6a9623f409878be921" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35895 } } }
I20260807 19:41:54.751200 874 tablet_replica.cc:442] T cc6c9d6cffef438e95dde8abd5c96771 P bca85bc96b104c22a05f824fdfcae087: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:41:54.758200 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.753163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49908 (local address 127.30.208.3:35895)
0807 19:41:54.753431 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:54.753437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:54.753490 (+ 53us) server_negotiation.cc:407] Connection header received
0807 19:41:54.753700 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:54.753704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:54.753770 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:54.753842 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:41:54.754319 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.755112 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.756484 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.756947 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.757050 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.757510 (+ 460us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:41:54.757618 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:41:54.757692 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:54.757850 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 19:41:54.757939 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260807 19:41:54.777060 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.771018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:37361 (local address 127.30.208.3:35895)
0807 19:41:54.771236 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:54.771241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:54.771903 (+ 662us) server_negotiation.cc:407] Connection header received
0807 19:41:54.772089 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:54.772095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:54.772180 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:54.772257 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:41:54.772775 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.773632 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.775213 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.775653 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.775777 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.776250 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:41:54.776372 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:41:54.776449 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:54.776561 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 19:41:54.776658 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":44}
I20260807 19:41:54.777136 894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.771054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:35895 (local address 127.30.208.3:37361)
0807 19:41:54.771797 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:54.771814 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:54.771966 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:54.772277 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:54.772281 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:54.772290 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:41:54.772618 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:54.772628 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.773764 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.773769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:54.775080 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:54.775094 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.775865 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.775872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:54.775982 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.776071 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:41:54.776463 (+ 392us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:41:54.776474 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 19:41:54.776546 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 19:41:54.776707 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":1,"thread_start_us":446,"threads_started":1}
I20260807 19:41:54.781981 832 consensus_queue.cc:1035] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bca85bc96b104c22a05f824fdfcae087" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 42339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:54.788511 832 consensus_queue.cc:1035] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa290f6d93644347af0c41cf053b09d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 33199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:54.854588 32753 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:41:54.854849 32753 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:41:54.858001 32753 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f34dfa46cc64fad92a117ae135de749: Sending DeleteTablet for 3 replicas of tablet bda7757230a34d05a6bcf8fd4b4cfe63
I20260807 19:41:54.858855 431 tablet_service.cc:1514] Processing DeleteTablet for tablet bda7757230a34d05a6bcf8fd4b4cfe63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:54 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:35332
I20260807 19:41:54.858989 581 tablet_service.cc:1514] Processing DeleteTablet for tablet bda7757230a34d05a6bcf8fd4b4cfe63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:54 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:44212
I20260807 19:41:54.860316 900 tablet_replica.cc:330] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3: stopping tablet replica
I20260807 19:41:54.860430 899 tablet_replica.cc:330] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087: stopping tablet replica
I20260807 19:41:54.860530 900 raft_consensus.cc:2229] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:41:54.860612 899 raft_consensus.cc:2229] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:41:54.860785 900 raft_consensus.cc:2258] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:54.860874 899 raft_consensus.cc:2258] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:54.860818 727 tablet_service.cc:1514] Processing DeleteTablet for tablet bda7757230a34d05a6bcf8fd4b4cfe63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:54 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:49892
I20260807 19:41:54.861989 900 ts_tablet_manager.cc:1905] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:54.862001 899 ts_tablet_manager.cc:1905] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:54.869161 900 ts_tablet_manager.cc:1918] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:41:54.869161 899 ts_tablet_manager.cc:1918] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:41:54.869349 899 log.cc:1196] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/bda7757230a34d05a6bcf8fd4b4cfe63
I20260807 19:41:54.869359 900 log.cc:1196] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/bda7757230a34d05a6bcf8fd4b4cfe63
I20260807 19:41:54.869884 900 ts_tablet_manager.cc:1939] T bda7757230a34d05a6bcf8fd4b4cfe63 P fa290f6d93644347af0c41cf053b09d3: Deleting consensus metadata
I20260807 19:41:54.869957 899 ts_tablet_manager.cc:1939] T bda7757230a34d05a6bcf8fd4b4cfe63 P bca85bc96b104c22a05f824fdfcae087: Deleting consensus metadata
I20260807 19:41:54.871637 32738 catalog_manager.cc:4841] TS fa290f6d93644347af0c41cf053b09d3 (127.30.208.2:33199): tablet bda7757230a34d05a6bcf8fd4b4cfe63 (table test-table [id=c56f78b354ba487786c741491ecf1634]) successfully deleted
I20260807 19:41:54.871948 32738 catalog_manager.cc:4841] TS bca85bc96b104c22a05f824fdfcae087 (127.30.208.1:42339): tablet bda7757230a34d05a6bcf8fd4b4cfe63 (table test-table [id=c56f78b354ba487786c741491ecf1634]) successfully deleted
I20260807 19:41:54.872740 901 tablet_replica.cc:330] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921: stopping tablet replica
I20260807 19:41:54.872946 901 raft_consensus.cc:2229] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:41:54.873652 901 raft_consensus.cc:2258] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:54.875057 901 ts_tablet_manager.cc:1905] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:54.882145 901 ts_tablet_manager.cc:1918] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:41:54.882321 901 log.cc:1196] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/bda7757230a34d05a6bcf8fd4b4cfe63
I20260807 19:41:54.882978 901 ts_tablet_manager.cc:1939] T bda7757230a34d05a6bcf8fd4b4cfe63 P 532f5ac5de224f6a9623f409878be921: Deleting consensus metadata
I20260807 19:41:54.884310 32741 catalog_manager.cc:4841] TS 532f5ac5de224f6a9623f409878be921 (127.30.208.3:35895): tablet bda7757230a34d05a6bcf8fd4b4cfe63 (table test-table [id=c56f78b354ba487786c741491ecf1634]) successfully deleted
Aug 07 19:41:54 dist-test-slave-vq2q krb5kdc[32707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131713, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:41:54.893108 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.873238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35348 (local address 127.30.208.1:42339)
0807 19:41:54.874217 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:54.874223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:54.874875 (+ 652us) server_negotiation.cc:407] Connection header received
0807 19:41:54.875073 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:54.875078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:54.875192 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:54.875302 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:54.877322 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.878519 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.880373 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.881094 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.881276 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.886788 (+ 5512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:54.886806 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:54.886823 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:54.886877 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:54.890105 (+ 3228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:54.891007 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:54.891015 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:54.891017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:54.891160 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:54.891536 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:54.891539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:54.891541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:54.891920 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:54.892150 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:54.892563 (+ 413us) server_negotiation.cc:299] Negotiation successful
0807 19:41:54.892774 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":841,"thread_start_us":902,"threads_started":1}
W20260807 19:41:54.893916 500 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35348
I20260807 19:41:54.911068 866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.897407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58132 (local address 127.30.208.62:35625)
0807 19:41:54.897601 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:54.897607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:54.897725 (+ 118us) server_negotiation.cc:407] Connection header received
0807 19:41:54.897878 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:54.897884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:54.898050 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:54.898158 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:54.899524 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.900616 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.902067 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.902796 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.902921 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.905057 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:54.905064 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:54.905069 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:54.905110 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:54.908437 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:54.909038 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:54.909044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:54.909048 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:54.909248 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:54.909571 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:54.909576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:54.909580 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:54.909984 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:54.910245 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:54.910499 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 19:41:54.910730 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 19:41:54.911746 32753 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58132
I20260807 19:41:54.915758 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 361
I20260807 19:41:54.929678 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.916661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35358 (local address 127.30.208.1:42339)
0807 19:41:54.916854 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:54.916860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:54.917017 (+ 157us) server_negotiation.cc:407] Connection header received
0807 19:41:54.917234 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:54.917238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:54.917304 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:54.917400 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:54.918800 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.919494 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.921239 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.921670 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.921763 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.924485 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:54.924490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:54.924493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:54.924525 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:54.927071 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:54.927818 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:54.927822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:54.927824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:54.927939 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:54.928307 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:54.928312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:54.928316 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:54.928585 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:54.928772 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:54.929167 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 19:41:54.929413 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 19:41:54.930342 500 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35358
W20260807 19:41:54.930912 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 361: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:41:54.931095 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 361
I20260807 19:41:54.956300 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 511
Aug 07 19:41:54 dist-test-slave-vq2q krb5kdc[32707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131713, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:41:54.972591 910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.957156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44228 (local address 127.30.208.2:33199)
0807 19:41:54.957719 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:54.957723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:54.957740 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:41:54.957792 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:54.957796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:54.957860 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:54.957954 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:54.959176 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.959939 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.961165 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:54.961706 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:54.961824 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:54.966000 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:54.966015 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:54.966029 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:54.966069 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:54.969546 (+ 3477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:54.970315 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:54.970325 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:54.970329 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:54.970525 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:54.970950 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:54.970955 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:54.970959 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:54.971423 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:54.971721 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:54.971952 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 19:41:54.972197 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":285,"threads_started":1}
W20260807 19:41:54.973366 648 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44228
W20260807 19:41:54.974001 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 511: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:41:54.974157 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 511
I20260807 19:41:54.997856 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 659
Aug 07 19:41:55 dist-test-slave-vq2q krb5kdc[32707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131713, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:41:55.018062 880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:54.998689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49912 (local address 127.30.208.3:35895)
0807 19:41:54.998889 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:54.998895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:54.998918 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:41:54.999036 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:54.999042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:54.999123 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:54.999249 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:55.000568 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:55.001713 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:55.003033 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:55.003744 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:55.003883 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:55.009518 (+ 5635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:55.009532 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:55.009549 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:55.009601 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:55.014084 (+ 4483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:55.015038 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:55.015048 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:55.015052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:55.015242 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:55.015670 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:55.015675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:55.015679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:55.016165 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:55.016483 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:55.017410 (+ 927us) server_negotiation.cc:299] Negotiation successful
0807 19:41:55.017654 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260807 19:41:55.018868 797 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49912
W20260807 19:41:55.019558 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 659: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:41:55.019735 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 659
I20260807 19:41:55.040566 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 32721
I20260807 19:41:55.041759 328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:41:55.157893 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 32721
2026-08-07T19:41:55Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4846 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:41:55 dist-test-slave-vq2q krb5kdc[914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:41:55 dist-test-slave-vq2q krb5kdc[914](info): set up 2 sockets
Aug 07 19:41:55 dist-test-slave-vq2q krb5kdc[914](info): commencing operation
krb5kdc: starting...
W20260807 19:41:56.207875 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.006s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:41:56 dist-test-slave-vq2q krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:41:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:41:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:41:56.404960 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33679
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:33429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:33679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:56.731391 927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:56.731712 927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:56.731856 927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:56.739133 927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:41:56.739259 927 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 19:41:56.739323 927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:56.739413 927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:56.739485 927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:41:56.739565 927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:41:56.749543 927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:33429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:33679
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:56.750898 927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:56.752228 927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:56.763972 932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:56.763895 935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:56.763917 933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:56.765098 927 server_base.cc:1034] running on GCE node
I20260807 19:41:56.765902 927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:56.767258 927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:56.768482 927 hybrid_clock.cc:648] HybridClock initialized: now 1786131716768417 us; error 32 us; skew 500 ppm
I20260807 19:41:56.768800 927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:56 dist-test-slave-vq2q krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131716, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:56.776899 927 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:41:56.778545 927 webserver.cc:460] Webserver started at http://127.30.208.62:45737/ using document root <none> and password file <none>
I20260807 19:41:56.779299 927 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:56.779410 927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:56.779742 927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:56.782881 927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "fae44f708a8f4556b4231a114013300e"
format_stamp: "Formatted at 2026-08-07 19:41:56 on dist-test-slave-vq2q"
server_key: "e7804db8853615f0cbf63483571b81ff"
server_key_iv: "28a6419ee68ef35f32e1eee67bff1099"
server_key_version: "encryptionkey@0"
I20260807 19:41:56.783667 927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "fae44f708a8f4556b4231a114013300e"
format_stamp: "Formatted at 2026-08-07 19:41:56 on dist-test-slave-vq2q"
server_key: "e7804db8853615f0cbf63483571b81ff"
server_key_iv: "28a6419ee68ef35f32e1eee67bff1099"
server_key_version: "encryptionkey@0"
I20260807 19:41:56.789103 927 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:41:56.792564 942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:56.793907 927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:41:56.794073 927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "fae44f708a8f4556b4231a114013300e"
format_stamp: "Formatted at 2026-08-07 19:41:56 on dist-test-slave-vq2q"
server_key: "e7804db8853615f0cbf63483571b81ff"
server_key_iv: "28a6419ee68ef35f32e1eee67bff1099"
server_key_version: "encryptionkey@0"
I20260807 19:41:56.794278 927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:56.819890 927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:56.824281 927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:56.824543 927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:56.844940 927 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:33679
I20260807 19:41:56.844959 1006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:33679 every 8 connection(s)
I20260807 19:41:56.846393 927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:41:56.850246 1007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:56.854189 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 927
I20260807 19:41:56.854382 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:41:56.855037 31552 external_mini_cluster.cc:1428] Setting key cdaa6792af1c3fdae1dc1ea97d31abd5
I20260807 19:41:56.862664 1007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e: Bootstrap starting.
I20260807 19:41:56.866494 1007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:56.867875 1007 log.cc:824] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:41:56 dist-test-slave-vq2q krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:56.871075 1007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e: No bootstrap required, opened a new log
I20260807 19:41:56.879599 1010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:56.857019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41578 (local address 127.30.208.62:33679)
0807 19:41:56.857861 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:56.857876 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:56.857928 (+ 52us) server_negotiation.cc:407] Connection header received
0807 19:41:56.860730 (+ 2802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:56.860764 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:56.861340 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:56.861799 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:56.863154 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:56.865165 (+ 2011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:56.866470 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:56.867532 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:56.867722 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:56.872313 (+ 4591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:56.872324 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:56.872339 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:56.872407 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:56.875487 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:56.876304 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:56.876311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:56.876316 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:56.876507 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:56.876887 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:56.876890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:56.876892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:56.877498 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:56.877718 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:56.878179 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 19:41:56.878565 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":356,"threads_started":1}
I20260807 19:41:56.879753 1007 raft_consensus.cc:348] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fae44f708a8f4556b4231a114013300e" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33679 } }
I20260807 19:41:56.880162 1007 raft_consensus.cc:374] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:56.880256 1007 raft_consensus.cc:729] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae44f708a8f4556b4231a114013300e, State: Initialized, Role: FOLLOWER
I20260807 19:41:56.880843 1007 consensus_queue.cc:260] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fae44f708a8f4556b4231a114013300e" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33679 } }
I20260807 19:41:56.881102 1007 raft_consensus.cc:388] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:41:56.881263 1007 raft_consensus.cc:482] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:41:56.881438 1007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:56.883657 1007 raft_consensus.cc:504] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fae44f708a8f4556b4231a114013300e" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33679 } }
I20260807 19:41:56.884193 1007 leader_election.cc:302] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fae44f708a8f4556b4231a114013300e; no voters:
I20260807 19:41:56.884799 1007 leader_election.cc:288] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:41:56.884927 1012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:41:56.885597 1012 raft_consensus.cc:686] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [term 1 LEADER]: Becoming Leader. State: Replica: fae44f708a8f4556b4231a114013300e, State: Running, Role: LEADER
I20260807 19:41:56.886128 1012 consensus_queue.cc:237] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fae44f708a8f4556b4231a114013300e" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33679 } }
I20260807 19:41:56.886824 1007 sys_catalog.cc:564] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:41:56.888852 1013 sys_catalog.cc:455] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fae44f708a8f4556b4231a114013300e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fae44f708a8f4556b4231a114013300e" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33679 } } }
I20260807 19:41:56.889276 1013 sys_catalog.cc:458] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [sys.catalog]: This master's current role is: LEADER
I20260807 19:41:56.889416 1014 sys_catalog.cc:455] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fae44f708a8f4556b4231a114013300e. Latest consensus state: current_term: 1 leader_uuid: "fae44f708a8f4556b4231a114013300e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fae44f708a8f4556b4231a114013300e" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33679 } } }
I20260807 19:41:56.889714 1014 sys_catalog.cc:458] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e [sys.catalog]: This master's current role is: LEADER
I20260807 19:41:56.891638 1018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:41:56.897217 1018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:41:56.906193 1018 catalog_manager.cc:1347] Generated new cluster ID: 4431d43a33834e5dbca8f0f487fc5a2d
I20260807 19:41:56.906325 1018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:41:56.927145 1018 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:41:56.928557 1018 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:41:56.939965 1018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e: Generated new TSK 0
I20260807 19:41:56.940878 1018 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:41:56.993122 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33679
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:33429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:57.325765 1034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:57.326064 1034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:57.326192 1034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:57.333808 1034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:57.333930 1034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:57.334055 1034 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 19:41:57.334116 1034 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:41:57.334199 1034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:41:57.344255 1034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:33429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.208.62:33679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:57.345645 1034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:57.347015 1034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:57.359874 1039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:57.359848 1043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:57.360009 1041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:57.361123 1034 server_base.cc:1034] running on GCE node
I20260807 19:41:57.361802 1034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:57.362751 1034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:57.363962 1034 hybrid_clock.cc:648] HybridClock initialized: now 1786131717363909 us; error 37 us; skew 500 ppm
I20260807 19:41:57.364302 1034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:57 dist-test-slave-vq2q krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:57.367516 1034 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:41:57.368973 1034 webserver.cc:460] Webserver started at http://127.30.208.1:37089/ using document root <none> and password file <none>
I20260807 19:41:57.369673 1034 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:57.369803 1034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:57.370100 1034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:57.372980 1034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "f97354f814c54d61a0f39aa2dc145d43"
format_stamp: "Formatted at 2026-08-07 19:41:57 on dist-test-slave-vq2q"
server_key: "328686de9b5cb5b2d6fa4c38e441e4e1"
server_key_iv: "229618ea3e05ea41b11d0b7e3af85544"
server_key_version: "encryptionkey@0"
I20260807 19:41:57.373766 1034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "f97354f814c54d61a0f39aa2dc145d43"
format_stamp: "Formatted at 2026-08-07 19:41:57 on dist-test-slave-vq2q"
server_key: "328686de9b5cb5b2d6fa4c38e441e4e1"
server_key_iv: "229618ea3e05ea41b11d0b7e3af85544"
server_key_version: "encryptionkey@0"
I20260807 19:41:57.378875 1034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:41:57.382378 1050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:57.383677 1034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:41:57.383859 1034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "f97354f814c54d61a0f39aa2dc145d43"
format_stamp: "Formatted at 2026-08-07 19:41:57 on dist-test-slave-vq2q"
server_key: "328686de9b5cb5b2d6fa4c38e441e4e1"
server_key_iv: "229618ea3e05ea41b11d0b7e3af85544"
server_key_version: "encryptionkey@0"
I20260807 19:41:57.384049 1034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:57.395164 1034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:57.399595 1034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:57.399890 1034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:57.403023 1034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:57.403151 1034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:57.403265 1034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:57.403357 1034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:41:57 dist-test-slave-vq2q krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:57.427495 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:57.403981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33679 (local address 127.30.208.1:47429)
0807 19:41:57.404770 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:57.404818 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:57.406755 (+ 1937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:57.407607 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:57.407623 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:57.408479 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:57.409395 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:57.409422 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.410988 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.410994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:57.413089 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:57.413109 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.414278 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.414286 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:57.414474 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:57.415169 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:57.415193 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:57.417942 (+ 2749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:57.422165 (+ 4223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:57.422179 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:57.422202 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:57.422725 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:57.423169 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:57.423175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:57.423179 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:57.423529 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:57.424289 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:57.424301 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:57.424826 (+ 525us) client_negotiation.cc:769] Sending connection context
0807 19:41:57.425217 (+ 391us) client_negotiation.cc:240] Negotiation successful
0807 19:41:57.425856 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":336,"threads_started":1}
I20260807 19:41:57.428552 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:57.403262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:47429 (local address 127.30.208.62:33679)
0807 19:41:57.403786 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:57.403793 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:57.404959 (+ 1166us) server_negotiation.cc:407] Connection header received
0807 19:41:57.407134 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:57.407141 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:57.407237 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:57.407374 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:57.409631 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.410781 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.413325 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.413973 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.414109 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:57.418256 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:57.418263 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:57.418269 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:57.418348 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:57.421998 (+ 3650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:57.422867 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:57.422872 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:57.422875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:57.423029 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:57.423688 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:57.423693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:57.423697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:57.424031 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:57.424195 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:57.428084 (+ 3889us) server_negotiation.cc:299] Negotiation successful
0807 19:41:57.428280 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":285,"threads_started":1}
I20260807 19:41:57.448372 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:57.434151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:39109 (local address 127.30.208.62:33679)
0807 19:41:57.434350 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:57.434357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:57.434927 (+ 570us) server_negotiation.cc:407] Connection header received
0807 19:41:57.435150 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:57.435156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:57.435244 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:57.435365 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:57.436611 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.437380 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.439481 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.440077 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.440243 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:57.442846 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:57.442854 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:57.442858 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:57.442896 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:57.445757 (+ 2861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:57.446620 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:57.446624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:57.446627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:57.446754 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:57.447106 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:57.447108 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:57.447110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:57.447381 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:57.447492 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:57.447945 (+ 453us) server_negotiation.cc:299] Negotiation successful
0807 19:41:57.448114 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:41:57.448779 1137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:57.434145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33679 (local address 127.30.208.1:39109)
0807 19:41:57.434800 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:57.434822 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:57.434992 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:57.435414 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:57.435420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:57.436043 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:57.436470 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:57.436484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.437513 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.437518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:57.439298 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:57.439316 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.440315 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.440323 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:57.440510 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:57.441225 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:57.441248 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:57.442636 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:57.445892 (+ 3256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:57.445899 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:57.445903 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:57.446513 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:57.446847 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:57.446853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:57.446857 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:57.447019 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:57.447475 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:57.447480 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:57.447697 (+ 217us) client_negotiation.cc:769] Sending connection context
0807 19:41:57.447868 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 19:41:57.448260 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":324,"threads_started":1}
I20260807 19:41:57.455307 1177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:40045 every 8 connection(s)
I20260807 19:41:57.455307 1034 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:40045
I20260807 19:41:57.456557 1034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:41:57.458663 1178 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33679
I20260807 19:41:57.458892 1178 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:57.459535 1178 heartbeater.cc:511] Master 127.30.208.62:33679 requested a full tablet report, sending...
I20260807 19:41:57.461570 960 ts_manager.cc:194] Registered new tserver with Master: f97354f814c54d61a0f39aa2dc145d43 (127.30.208.1:40045)
I20260807 19:41:57.463037 960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:47429
I20260807 19:41:57.466066 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 1034
I20260807 19:41:57.466236 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:41:57.466809 31552 external_mini_cluster.cc:1428] Setting key 18acacf4b1769f98fcd06612ce6bcecb
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:41:57.512590 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33679
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:33429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:57.843104 1183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:57.843411 1183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:57.843518 1183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:57.851043 1183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:57.851184 1183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:57.851310 1183 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 19:41:57.851379 1183 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:41:57.851490 1183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:41:57.861693 1183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:33429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.208.62:33679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:57.863076 1183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:57.864416 1183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:57.877354 1189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:57.877385 1190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:57.877353 1192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:57.878288 1183 server_base.cc:1034] running on GCE node
I20260807 19:41:57.878935 1183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:57.879855 1183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:57.881062 1183 hybrid_clock.cc:648] HybridClock initialized: now 1786131717881016 us; error 36 us; skew 500 ppm
I20260807 19:41:57.881394 1183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:57 dist-test-slave-vq2q krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:57.884578 1183 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:41:57.886135 1183 webserver.cc:460] Webserver started at http://127.30.208.2:40065/ using document root <none> and password file <none>
I20260807 19:41:57.886839 1183 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:57.886967 1183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:57.887321 1183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:57.890450 1183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "ab40e6ac7585412cbf265a8bd66aa72d"
format_stamp: "Formatted at 2026-08-07 19:41:57 on dist-test-slave-vq2q"
server_key: "a948a67d9355f63187de3c5233ac2d85"
server_key_iv: "6b5c2f1dea579811abc55d7fac130cca"
server_key_version: "encryptionkey@0"
I20260807 19:41:57.891225 1183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "ab40e6ac7585412cbf265a8bd66aa72d"
format_stamp: "Formatted at 2026-08-07 19:41:57 on dist-test-slave-vq2q"
server_key: "a948a67d9355f63187de3c5233ac2d85"
server_key_iv: "6b5c2f1dea579811abc55d7fac130cca"
server_key_version: "encryptionkey@0"
I20260807 19:41:57.896736 1183 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:41:57.900615 1199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:57.901990 1183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:41:57.902134 1183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "ab40e6ac7585412cbf265a8bd66aa72d"
format_stamp: "Formatted at 2026-08-07 19:41:57 on dist-test-slave-vq2q"
server_key: "a948a67d9355f63187de3c5233ac2d85"
server_key_iv: "6b5c2f1dea579811abc55d7fac130cca"
server_key_version: "encryptionkey@0"
I20260807 19:41:57.902309 1183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:57.940128 1183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:57.944619 1183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:57.944921 1183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:57.947628 1183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:57.947748 1183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:57.947870 1183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:57.947943 1183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:41:57 dist-test-slave-vq2q krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131717, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:57.969735 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:57.948386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:59759 (local address 127.30.208.62:33679)
0807 19:41:57.948605 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:57.948611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:57.949957 (+ 1346us) server_negotiation.cc:407] Connection header received
0807 19:41:57.952033 (+ 2076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:57.952039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:57.952121 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:57.952210 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:57.954406 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.955435 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.958005 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.958458 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.958575 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:57.963282 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:57.963290 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:57.963295 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:57.963340 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:57.966410 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:57.967385 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:57.967390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:57.967394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:57.967517 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:57.967971 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:57.967975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:57.967977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:57.968265 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:57.968400 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:57.969130 (+ 730us) server_negotiation.cc:299] Negotiation successful
0807 19:41:57.969390 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:41:57.970810 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:57.948916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33679 (local address 127.30.208.2:59759)
0807 19:41:57.949770 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:57.949813 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:57.951744 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:57.952463 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:57.952471 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:57.953253 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:57.954149 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:57.954180 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.955599 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.955605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:57.957841 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:57.957865 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.958738 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.958746 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:57.959067 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:57.959800 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:57.959845 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:57.962997 (+ 3152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:57.966579 (+ 3582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:57.966589 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:57.966607 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:57.967205 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:57.967628 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:57.967633 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:57.967637 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:57.967849 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:57.968400 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:57.968408 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:57.968771 (+ 363us) client_negotiation.cc:769] Sending connection context
0807 19:41:57.969012 (+ 241us) client_negotiation.cc:240] Negotiation successful
0807 19:41:57.969588 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":369,"threads_started":1}
I20260807 19:41:57.990808 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:57.976584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:39401 (local address 127.30.208.62:33679)
0807 19:41:57.976737 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:57.976741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:57.977473 (+ 732us) server_negotiation.cc:407] Connection header received
0807 19:41:57.977645 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:57.977649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:57.977709 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:57.977791 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:57.978765 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.979492 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.981163 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.981756 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.981866 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:57.984626 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:57.984633 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:57.984638 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:57.984680 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:57.987921 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:57.988782 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:57.988789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:57.988793 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:57.988979 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:57.989471 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:57.989476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:57.989480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:57.989811 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:57.989930 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:57.990407 (+ 477us) server_negotiation.cc:299] Negotiation successful
0807 19:41:57.990578 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 19:41:57.991195 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:57.976651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33679 (local address 127.30.208.2:39401)
0807 19:41:57.977365 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:57.977384 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:57.977531 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:57.977831 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:57.977836 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:57.978328 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:57.978624 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:57.978633 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.979624 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.979630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:57.981021 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:57.981045 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:57.982126 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:57.982136 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:57.982343 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:57.983069 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:57.983097 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:57.984413 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:57.988049 (+ 3636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:57.988055 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:57.988059 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:57.988660 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:57.989109 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:57.989114 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:57.989118 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:57.989348 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:57.989920 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:57.989925 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:57.990149 (+ 224us) client_negotiation.cc:769] Sending connection context
0807 19:41:57.990318 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 19:41:57.990687 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":370,"threads_started":1}
I20260807 19:41:57.999642 1183 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:41287
I20260807 19:41:57.999725 1325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:41287 every 8 connection(s)
I20260807 19:41:58.000939 1183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:41:58.002696 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 1183
I20260807 19:41:58.002853 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:41:58.003073 1326 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33679
I20260807 19:41:58.003286 1326 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:58.003325 31552 external_mini_cluster.cc:1428] Setting key 83628c57b97fdc1badf41678198607af
I20260807 19:41:58.004071 1326 heartbeater.cc:511] Master 127.30.208.62:33679 requested a full tablet report, sending...
I20260807 19:41:58.005810 960 ts_manager.cc:194] Registered new tserver with Master: ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287)
I20260807 19:41:58.006651 960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:59759
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:41:58.049407 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33679
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:33429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:41:58.389623 1330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:41:58.389920 1330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:41:58.390033 1330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:41:58.397403 1330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:41:58.397522 1330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:41:58.397626 1330 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 19:41:58.397710 1330 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:41:58.397812 1330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:41:58.408788 1330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:33429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.208.62:33679
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:41:58.410247 1330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:41:58.411624 1330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:41:58.425056 1337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:58.424997 1335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:41:58.425027 1339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:41:58.430024 1330 server_base.cc:1034] running on GCE node
I20260807 19:41:58.430788 1330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:41:58.431803 1330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:41:58.433029 1330 hybrid_clock.cc:648] HybridClock initialized: now 1786131718432990 us; error 50 us; skew 500 ppm
I20260807 19:41:58.433384 1330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:41:58 dist-test-slave-vq2q krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131718, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:41:58.436595 1330 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:41:58.438196 1330 webserver.cc:460] Webserver started at http://127.30.208.3:37235/ using document root <none> and password file <none>
I20260807 19:41:58.438949 1330 fs_manager.cc:359] Metadata directory not provided
I20260807 19:41:58.439079 1330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:41:58.439388 1330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:41:58.442456 1330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "962ba2e327344703bf1ca14efb11ec49"
format_stamp: "Formatted at 2026-08-07 19:41:58 on dist-test-slave-vq2q"
server_key: "97ba4ca85ec6606c0e7c2764fb22ac26"
server_key_iv: "f1f443da09670687f9423ccab91106e8"
server_key_version: "encryptionkey@0"
I20260807 19:41:58.443203 1330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "962ba2e327344703bf1ca14efb11ec49"
format_stamp: "Formatted at 2026-08-07 19:41:58 on dist-test-slave-vq2q"
server_key: "97ba4ca85ec6606c0e7c2764fb22ac26"
server_key_iv: "f1f443da09670687f9423ccab91106e8"
server_key_version: "encryptionkey@0"
I20260807 19:41:58.448235 1330 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 19:41:58.451608 1347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:58.452984 1330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:41:58.453155 1330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "962ba2e327344703bf1ca14efb11ec49"
format_stamp: "Formatted at 2026-08-07 19:41:58 on dist-test-slave-vq2q"
server_key: "97ba4ca85ec6606c0e7c2764fb22ac26"
server_key_iv: "f1f443da09670687f9423ccab91106e8"
server_key_version: "encryptionkey@0"
I20260807 19:41:58.453409 1330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:41:58.465762 1178 heartbeater.cc:503] Master 127.30.208.62:33679 was elected leader, sending a full tablet report...
I20260807 19:41:58.473014 1330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:41:58.477108 1330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:41:58.477411 1330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:41:58.480206 1330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:41:58.480331 1330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:41:58.480453 1330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:41:58.480533 1330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:41:58 dist-test-slave-vq2q krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131718, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:58.503517 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.480933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:48263 (local address 127.30.208.62:33679)
0807 19:41:58.481135 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:58.481140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:58.482928 (+ 1788us) server_negotiation.cc:407] Connection header received
0807 19:41:58.485029 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:58.485036 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:58.485131 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:58.485293 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:58.487863 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.488856 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.491123 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.491576 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.491721 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.496736 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:58.496742 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:58.496745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:58.496784 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:58.499603 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:58.500580 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:58.500585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:58.500587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:58.500727 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:58.501229 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:58.501234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:58.501236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:58.501545 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:58.501702 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:58.502907 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0807 19:41:58.503199 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:41:58.505118 1359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.481902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33679 (local address 127.30.208.3:48263)
0807 19:41:58.482771 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:58.482808 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:58.484708 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:58.485756 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:58.485773 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:58.486680 (+ 907us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:58.487654 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.487707 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.489023 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.489029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.490976 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.490993 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.492071 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.492079 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.492362 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.493152 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:58.493212 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:58.496462 (+ 3250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:58.499745 (+ 3283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:58.499755 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:58.499775 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:58.500440 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:58.500825 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:58.500830 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:58.500834 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:58.501082 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:58.501687 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:58.501701 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:58.502306 (+ 605us) client_negotiation.cc:769] Sending connection context
0807 19:41:58.502735 (+ 429us) client_negotiation.cc:240] Negotiation successful
0807 19:41:58.503421 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":368,"threads_started":1}
I20260807 19:41:58.526710 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.511492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:45415 (local address 127.30.208.62:33679)
0807 19:41:58.511652 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:58.511657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:58.512411 (+ 754us) server_negotiation.cc:407] Connection header received
0807 19:41:58.512653 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:58.512658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:58.512728 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:58.512816 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:58.514239 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.515208 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.517151 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.517736 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.517856 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.520259 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:58.520265 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:58.520269 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:58.520313 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:58.523562 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:58.524524 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:58.524529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:58.524533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:58.524704 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:58.525099 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:58.525102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:58.525104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:58.525421 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:58.525540 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:58.526229 (+ 689us) server_negotiation.cc:299] Negotiation successful
0807 19:41:58.526442 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:41:58.527004 1409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.511555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33679 (local address 127.30.208.3:45415)
0807 19:41:58.512274 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:58.512298 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:58.512498 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:58.512872 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:58.512877 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:58.513601 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:58.514028 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.514047 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.515376 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.515380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.517015 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.517030 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.517960 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.517967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.518160 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.518907 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:58.518929 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:58.520015 (+ 1086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:58.523725 (+ 3710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:58.523732 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:58.523738 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:58.524390 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:58.524820 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:58.524825 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:58.524828 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:58.525000 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:58.525533 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:58.525538 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:58.525784 (+ 246us) client_negotiation.cc:769] Sending connection context
0807 19:41:58.526092 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 19:41:58.526503 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":395,"threads_started":1}
I20260807 19:41:58.533933 1330 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:34923
I20260807 19:41:58.534010 1475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:34923 every 8 connection(s)
I20260807 19:41:58.535197 1330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:41:58.537433 1476 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33679
I20260807 19:41:58.537648 1476 heartbeater.cc:464] Registering TS with master...
I20260807 19:41:58.538287 1476 heartbeater.cc:511] Master 127.30.208.62:33679 requested a full tablet report, sending...
I20260807 19:41:58.539630 960 ts_manager.cc:194] Registered new tserver with Master: 962ba2e327344703bf1ca14efb11ec49 (127.30.208.3:34923)
I20260807 19:41:58.540467 960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:48263
I20260807 19:41:58.540741 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 1330
I20260807 19:41:58.540956 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:41:58.541546 31552 external_mini_cluster.cc:1428] Setting key bd90668274ec4a4624560d4ed108860c
I20260807 19:41:58.544541 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:41:58 dist-test-slave-vq2q krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:41:58 dist-test-slave-vq2q krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:58.593564 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.580170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41588 (local address 127.30.208.62:33679)
0807 19:41:58.580361 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:58.580367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:58.580517 (+ 150us) server_negotiation.cc:407] Connection header received
0807 19:41:58.580710 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:58.580715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:58.580782 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:58.580877 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:58.582305 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.583055 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.584285 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.584716 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.584833 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.588741 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:58.588746 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:58.588749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:58.588784 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:58.591185 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:58.591784 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:58.591788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:58.591791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:58.591930 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:58.592296 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:58.592300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:58.592302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:58.592590 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:58.592776 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:58.593034 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 19:41:58.593270 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:41:58.597461 960 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:41:58.604256 960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:41:58.638648 1493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.617378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47006 (local address 127.30.208.2:41287)
0807 19:41:58.630302 (+ 12924us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:58.630312 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:58.631258 (+ 946us) server_negotiation.cc:407] Connection header received
0807 19:41:58.631358 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:58.631365 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:58.631574 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:58.631752 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:58.632729 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.634198 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.636334 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.637499 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.637628 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.637831 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:58.638178 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 19:41:58.638344 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12751,"thread_start_us":12941,"threads_started":1}
I20260807 19:41:58.638711 1491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.616961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:41287 (local address 127.0.0.1:47006)
0807 19:41:58.630977 (+ 14016us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:58.630995 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:58.631122 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:58.631963 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:58.631971 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:58.632017 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:58.632481 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.632498 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.634344 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.634350 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.636148 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.636163 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.637692 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.637700 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.637973 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.638011 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:41:58.638117 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 19:41:58.638304 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13882,"mutex_wait_us":71,"spinlock_wait_cycles":1792,"thread_start_us":13720,"threads_started":1}
I20260807 19:41:58.639937 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.616568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:34923 (local address 127.0.0.1:57518)
0807 19:41:58.630399 (+ 13831us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:58.630439 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:58.630670 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:58.631963 (+ 1293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:58.631971 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:58.632026 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:58.632931 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.632945 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.634655 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.634660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.636793 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.636808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.639127 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.639133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.639313 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.639346 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:41:58.639441 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 19:41:58.639566 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13572,"mutex_wait_us":339,"spinlock_wait_cycles":15360,"thread_start_us":13403,"threads_started":1}
I20260807 19:41:58.640734 1490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.616791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57518 (local address 127.30.208.3:34923)
0807 19:41:58.629843 (+ 13052us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:58.629854 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:58.630934 (+ 1080us) server_negotiation.cc:407] Connection header received
0807 19:41:58.631053 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:58.631061 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:58.631414 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:58.631606 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:58.633195 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.634515 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.636977 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.638401 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.638507 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.638724 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:58.639501 (+ 777us) server_negotiation.cc:299] Negotiation successful
0807 19:41:58.639666 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12817,"thread_start_us":12606,"threads_started":1}
I20260807 19:41:58.641149 1494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.631098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:40045 (local address 127.0.0.1:59570)
0807 19:41:58.632943 (+ 1845us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:58.632961 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:58.633080 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:58.633938 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:58.633943 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:58.633966 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:58.634349 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.634361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.636203 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.636208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.638355 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.638372 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.640403 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.640407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.640550 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.640585 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:41:58.640672 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 19:41:58.640802 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1713,"mutex_wait_us":1,"thread_start_us":1499,"threads_started":1}
I20260807 19:41:58.641777 1495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.631151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.30.208.1:40045)
0807 19:41:58.632458 (+ 1307us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:58.632465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:58.633248 (+ 783us) server_negotiation.cc:407] Connection header received
0807 19:41:58.633345 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:58.633351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:58.633548 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:58.633722 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:58.634537 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.636021 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.638561 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.639953 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.640080 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.640266 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:58.641440 (+ 1174us) server_negotiation.cc:299] Negotiation successful
0807 19:41:58.641565 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1148,"thread_start_us":992,"threads_started":1}
I20260807 19:41:58.642907 1254 tablet_service.cc:1467] Processing CreateTablet for tablet fb43c8f5e5544deb9eb42451e5dcd3d1 (DEFAULT_TABLE table=test-table [id=ab718851c7e44d1fa70479aa8e1a8a61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:58.644093 1254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb43c8f5e5544deb9eb42451e5dcd3d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:58.644224 1401 tablet_service.cc:1467] Processing CreateTablet for tablet fb43c8f5e5544deb9eb42451e5dcd3d1 (DEFAULT_TABLE table=test-table [id=ab718851c7e44d1fa70479aa8e1a8a61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:58.645723 1401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb43c8f5e5544deb9eb42451e5dcd3d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:58.645241 1106 tablet_service.cc:1467] Processing CreateTablet for tablet fb43c8f5e5544deb9eb42451e5dcd3d1 (DEFAULT_TABLE table=test-table [id=ab718851c7e44d1fa70479aa8e1a8a61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:58.646510 1106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb43c8f5e5544deb9eb42451e5dcd3d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:58.656626 1496 tablet_bootstrap.cc:492] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d: Bootstrap starting.
I20260807 19:41:58.659641 1496 tablet_bootstrap.cc:654] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:58.660632 1496 log.cc:824] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:58.661548 1498 tablet_bootstrap.cc:492] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43: Bootstrap starting.
I20260807 19:41:58.663693 1497 tablet_bootstrap.cc:492] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49: Bootstrap starting.
I20260807 19:41:58.665141 1498 tablet_bootstrap.cc:654] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:58.665777 1496 tablet_bootstrap.cc:492] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d: No bootstrap required, opened a new log
I20260807 19:41:58.666018 1496 ts_tablet_manager.cc:1397] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260807 19:41:58.666435 1498 log.cc:824] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:58.667152 1497 tablet_bootstrap.cc:654] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:58.668249 1497 log.cc:824] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49: Log is configured to *not* fsync() on all Append() calls
I20260807 19:41:58.669440 1498 tablet_bootstrap.cc:492] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43: No bootstrap required, opened a new log
I20260807 19:41:58.669705 1498 ts_tablet_manager.cc:1397] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 19:41:58.671240 1497 tablet_bootstrap.cc:492] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49: No bootstrap required, opened a new log
I20260807 19:41:58.671489 1497 ts_tablet_manager.cc:1397] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260807 19:41:58.674336 1496 raft_consensus.cc:348] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } }
I20260807 19:41:58.674700 1496 raft_consensus.cc:374] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:58.674831 1496 raft_consensus.cc:729] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab40e6ac7585412cbf265a8bd66aa72d, State: Initialized, Role: FOLLOWER
I20260807 19:41:58.675400 1496 consensus_queue.cc:260] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } }
I20260807 19:41:58.676508 1326 heartbeater.cc:503] Master 127.30.208.62:33679 was elected leader, sending a full tablet report...
I20260807 19:41:58.677002 1496 ts_tablet_manager.cc:1428] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260807 19:41:58.681520 1498 raft_consensus.cc:348] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } }
I20260807 19:41:58.682090 1498 raft_consensus.cc:374] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:58.682286 1498 raft_consensus.cc:729] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97354f814c54d61a0f39aa2dc145d43, State: Initialized, Role: FOLLOWER
I20260807 19:41:58.682160 1497 raft_consensus.cc:348] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } }
I20260807 19:41:58.682662 1497 raft_consensus.cc:374] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:58.682783 1497 raft_consensus.cc:729] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 962ba2e327344703bf1ca14efb11ec49, State: Initialized, Role: FOLLOWER
I20260807 19:41:58.683111 1498 consensus_queue.cc:260] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } }
I20260807 19:41:58.683334 1497 consensus_queue.cc:260] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } }
I20260807 19:41:58.684996 1498 ts_tablet_manager.cc:1428] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260807 19:41:58.684990 1497 ts_tablet_manager.cc:1428] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
I20260807 19:41:58.685305 1476 heartbeater.cc:503] Master 127.30.208.62:33679 was elected leader, sending a full tablet report...
W20260807 19:41:58.708847 1180 tablet.cc:2369] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:41:58.724632 1505 raft_consensus.cc:482] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:41:58.724962 1505 raft_consensus.cc:504] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } }
I20260807 19:41:58.726740 1505 leader_election.cc:288] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287), f97354f814c54d61a0f39aa2dc145d43 (127.30.208.1:40045)
I20260807 19:41:58.732684 1359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.727245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:41287 (local address 127.30.208.3:38655)
0807 19:41:58.727510 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:58.727533 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:58.727669 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:58.728045 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:58.728049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:58.728073 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:58.728245 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.728253 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.729251 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.729257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.730642 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.730657 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.732118 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.732124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.732268 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.732294 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 19:41:58.732379 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 19:41:58.732473 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":1}
I20260807 19:41:58.732803 1493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.727269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:38655 (local address 127.30.208.2:41287)
0807 19:41:58.727616 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:58.727621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:58.727644 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:41:58.727824 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:58.727829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:58.727909 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:58.728021 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:58.728393 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.729100 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.730791 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.731795 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.731962 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.732033 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:58.732421 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 19:41:58.732552 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211}
I20260807 19:41:58.733629 1495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.727201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:58891 (local address 127.30.208.1:40045)
0807 19:41:58.727429 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:58.727436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:58.728194 (+ 758us) server_negotiation.cc:407] Connection header received
0807 19:41:58.728389 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:58.728394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:58.728456 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:58.728528 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:58.729021 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.729763 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.731639 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.732658 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.732842 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.732912 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:58.733249 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 19:41:58.733393 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:41:58.733714 1507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.727278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:40045 (local address 127.30.208.3:58891)
0807 19:41:58.728096 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:58.728117 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:58.728252 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:58.728556 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:58.728560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:58.728585 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:58.728877 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.728886 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.729911 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.729915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.731466 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.731481 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.732894 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.732902 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.733076 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.733107 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 19:41:58.733223 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 19:41:58.733349 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":17,"thread_start_us":436,"threads_started":1}
I20260807 19:41:58.733760 1280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb43c8f5e5544deb9eb42451e5dcd3d1" candidate_uuid: "962ba2e327344703bf1ca14efb11ec49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" is_pre_election: true
I20260807 19:41:58.734306 1280 raft_consensus.cc:2454] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 962ba2e327344703bf1ca14efb11ec49 in term 0.
I20260807 19:41:58.734836 1127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb43c8f5e5544deb9eb42451e5dcd3d1" candidate_uuid: "962ba2e327344703bf1ca14efb11ec49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f97354f814c54d61a0f39aa2dc145d43" is_pre_election: true
I20260807 19:41:58.735136 1349 leader_election.cc:302] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 962ba2e327344703bf1ca14efb11ec49, ab40e6ac7585412cbf265a8bd66aa72d; no voters:
I20260807 19:41:58.735298 1127 raft_consensus.cc:2454] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 962ba2e327344703bf1ca14efb11ec49 in term 0.
I20260807 19:41:58.735531 1505 raft_consensus.cc:2781] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:41:58.735675 1505 raft_consensus.cc:482] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:41:58.735797 1505 raft_consensus.cc:3037] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:58.738133 1505 raft_consensus.cc:504] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } }
I20260807 19:41:58.738732 1505 leader_election.cc:288] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [CANDIDATE]: Term 1 election: Requested vote from peers ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287), f97354f814c54d61a0f39aa2dc145d43 (127.30.208.1:40045)
I20260807 19:41:58.739174 1280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb43c8f5e5544deb9eb42451e5dcd3d1" candidate_uuid: "962ba2e327344703bf1ca14efb11ec49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab40e6ac7585412cbf265a8bd66aa72d"
I20260807 19:41:58.739188 1127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb43c8f5e5544deb9eb42451e5dcd3d1" candidate_uuid: "962ba2e327344703bf1ca14efb11ec49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f97354f814c54d61a0f39aa2dc145d43"
I20260807 19:41:58.739353 1127 raft_consensus.cc:3037] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:58.739359 1280 raft_consensus.cc:3037] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:58.741616 1280 raft_consensus.cc:2454] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 962ba2e327344703bf1ca14efb11ec49 in term 1.
I20260807 19:41:58.741688 1127 raft_consensus.cc:2454] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 962ba2e327344703bf1ca14efb11ec49 in term 1.
I20260807 19:41:58.742004 1349 leader_election.cc:302] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 962ba2e327344703bf1ca14efb11ec49, ab40e6ac7585412cbf265a8bd66aa72d; no voters:
I20260807 19:41:58.742285 1505 raft_consensus.cc:2781] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:41:58.742722 1505 raft_consensus.cc:686] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [term 1 LEADER]: Becoming Leader. State: Replica: 962ba2e327344703bf1ca14efb11ec49, State: Running, Role: LEADER
I20260807 19:41:58.743273 1505 consensus_queue.cc:237] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } }
I20260807 19:41:58.747814 960 catalog_manager.cc:5495] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 962ba2e327344703bf1ca14efb11ec49 (127.30.208.3). New cstate: current_term: 1 leader_uuid: "962ba2e327344703bf1ca14efb11ec49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } health_report { overall_health: UNKNOWN } } }
W20260807 19:41:58.752918 1327 tablet.cc:2369] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:41:58.786837 1477 tablet.cc:2369] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 19:41:58 dist-test-slave-vq2q krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131716, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:41:58.797636 1515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.780761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33679 (local address 127.0.0.1:41592)
0807 19:41:58.781418 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:58.781433 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:58.781563 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:58.781938 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:58.781942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:58.782901 (+ 959us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:41:58.783275 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.783291 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.784992 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.784997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.786084 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:58.786097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.787027 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.787033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:58.787212 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.787945 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:41:58.787969 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:41:58.790897 (+ 2928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:41:58.794604 (+ 3707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:58.794617 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:58.794623 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:58.795182 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:58.795637 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:41:58.795640 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:41:58.795643 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:41:58.795794 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:41:58.796442 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:41:58.796450 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:41:58.796650 (+ 200us) client_negotiation.cc:769] Sending connection context
0807 19:41:58.796822 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 19:41:58.797133 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":331,"threads_started":1}
I20260807 19:41:58.797713 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:58.780750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41592 (local address 127.30.208.62:33679)
0807 19:41:58.780965 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:58.780970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:58.781514 (+ 544us) server_negotiation.cc:407] Connection header received
0807 19:41:58.781672 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:58.781678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:58.781769 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:58.781873 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:41:58.783851 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.784837 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.786218 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:58.786676 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:58.786789 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:58.791138 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:41:58.791145 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:41:58.791150 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:41:58.791188 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:41:58.794451 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:58.795331 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:58.795338 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:58.795342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:58.795526 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:41:58.795911 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:41:58.795917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:41:58.795919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:41:58.796301 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:41:58.796466 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:58.796970 (+ 504us) server_negotiation.cc:299] Negotiation successful
0807 19:41:58.797328 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":70}
I20260807 19:41:58.803207 960 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:41592:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 19:41:58.804242 960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:41:58.811038 1254 tablet_service.cc:1467] Processing CreateTablet for tablet cf8b99067b814c238b7090ba65a45618 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba8e55015620431ab1a6d94637e8e15b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:41:58.810981 1106 tablet_service.cc:1467] Processing CreateTablet for tablet cf8b99067b814c238b7090ba65a45618 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba8e55015620431ab1a6d94637e8e15b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:41:58.811899 1254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf8b99067b814c238b7090ba65a45618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:58.811468 1401 tablet_service.cc:1467] Processing CreateTablet for tablet cf8b99067b814c238b7090ba65a45618 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ba8e55015620431ab1a6d94637e8e15b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:41:58.811976 1106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf8b99067b814c238b7090ba65a45618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:58.812264 1401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf8b99067b814c238b7090ba65a45618. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:58.817215 1496 tablet_bootstrap.cc:492] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d: Bootstrap starting.
I20260807 19:41:58.818751 1496 tablet_bootstrap.cc:654] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:58.819222 1498 tablet_bootstrap.cc:492] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43: Bootstrap starting.
I20260807 19:41:58.820509 1497 tablet_bootstrap.cc:492] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49: Bootstrap starting.
I20260807 19:41:58.820880 1498 tablet_bootstrap.cc:654] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:58.821910 1496 tablet_bootstrap.cc:492] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d: No bootstrap required, opened a new log
I20260807 19:41:58.822067 1496 ts_tablet_manager.cc:1397] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 19:41:58.822198 1497 tablet_bootstrap.cc:654] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:58.822839 1496 raft_consensus.cc:348] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:58.823021 1496 raft_consensus.cc:374] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:58.823079 1496 raft_consensus.cc:729] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab40e6ac7585412cbf265a8bd66aa72d, State: Initialized, Role: FOLLOWER
I20260807 19:41:58.823434 1498 tablet_bootstrap.cc:492] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43: No bootstrap required, opened a new log
I20260807 19:41:58.823647 1498 ts_tablet_manager.cc:1397] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 19:41:58.823540 1496 consensus_queue.cc:260] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:58.824251 1496 ts_tablet_manager.cc:1428] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:41:58.824149 1503 tablet_replica.cc:439] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } }
I20260807 19:41:58.824565 1503 tablet_replica.cc:442] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:41:58.824692 1497 tablet_bootstrap.cc:492] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49: No bootstrap required, opened a new log
I20260807 19:41:58.824844 1497 ts_tablet_manager.cc:1397] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260807 19:41:58.825682 1497 raft_consensus.cc:348] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:58.825966 1497 raft_consensus.cc:374] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:58.825882 1498 raft_consensus.cc:348] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:58.826057 1497 raft_consensus.cc:729] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 962ba2e327344703bf1ca14efb11ec49, State: Initialized, Role: FOLLOWER
I20260807 19:41:58.826123 1498 raft_consensus.cc:374] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:58.826239 1498 raft_consensus.cc:729] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97354f814c54d61a0f39aa2dc145d43, State: Initialized, Role: FOLLOWER
I20260807 19:41:58.826359 1497 consensus_queue.cc:260] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:58.826519 1498 consensus_queue.cc:260] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:58.826824 1505 tablet_replica.cc:439] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } }
I20260807 19:41:58.827023 1497 ts_tablet_manager.cc:1428] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:41:58.827143 1505 tablet_replica.cc:442] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:41:58.827006 1504 tablet_replica.cc:439] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } }
I20260807 19:41:58.827404 1504 tablet_replica.cc:442] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:41:58.828974 1498 ts_tablet_manager.cc:1428] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 19:41:59.052579 1504 raft_consensus.cc:482] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:41:59.052838 1504 raft_consensus.cc:504] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:59.054565 1504 leader_election.cc:288] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287), 962ba2e327344703bf1ca14efb11ec49 (127.30.208.3:34923)
I20260807 19:41:59.061830 1493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:59.055094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:52741 (local address 127.30.208.2:41287)
0807 19:41:59.055287 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:59.055292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:59.055938 (+ 646us) server_negotiation.cc:407] Connection header received
0807 19:41:59.056112 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:59.056116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:59.056185 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:59.056270 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:59.056916 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.057741 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.059920 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.060841 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.061003 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:59.061080 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:59.061403 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 19:41:59.061533 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:41:59.061964 1490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:59.054986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:39225 (local address 127.30.208.3:34923)
0807 19:41:59.055178 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:59.055185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:59.055834 (+ 649us) server_negotiation.cc:407] Connection header received
0807 19:41:59.056028 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:59.056032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:59.056095 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:59.056182 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:41:59.056856 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.057703 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.060104 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.060917 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.061024 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:59.061094 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:59.061587 (+ 493us) server_negotiation.cc:299] Negotiation successful
0807 19:41:59.061720 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:41:59.062033 1521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:59.055037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:41287 (local address 127.30.208.1:52741)
0807 19:41:59.055816 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:59.055830 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:59.055974 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:59.056318 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:59.056323 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:59.056358 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:59.056711 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:59.056719 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.057890 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.057899 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:59.059705 (+ 1806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:59.059721 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.061060 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.061066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:59.061241 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:59.061271 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 19:41:59.061358 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 19:41:59.061546 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":4,"thread_start_us":443,"threads_started":1}
I20260807 19:41:59.062103 1520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:59.054964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:34923 (local address 127.30.208.1:39225)
0807 19:41:59.055716 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:59.055737 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:59.055893 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:59.056208 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:59.056212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:59.056237 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:41:59.056658 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:59.056670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.057828 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.057834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:59.059911 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:59.059927 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.061314 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.061319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:59.061433 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:59.061456 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:41:59.061534 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 19:41:59.061652 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"mutex_wait_us":85,"thread_start_us":443,"threads_started":1}
I20260807 19:41:59.062803 1280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf8b99067b814c238b7090ba65a45618" candidate_uuid: "f97354f814c54d61a0f39aa2dc145d43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" is_pre_election: true
I20260807 19:41:59.062939 1427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf8b99067b814c238b7090ba65a45618" candidate_uuid: "f97354f814c54d61a0f39aa2dc145d43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "962ba2e327344703bf1ca14efb11ec49" is_pre_election: true
I20260807 19:41:59.063040 1280 raft_consensus.cc:2454] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f97354f814c54d61a0f39aa2dc145d43 in term 0.
I20260807 19:41:59.063279 1427 raft_consensus.cc:2454] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f97354f814c54d61a0f39aa2dc145d43 in term 0.
I20260807 19:41:59.063525 1053 leader_election.cc:302] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab40e6ac7585412cbf265a8bd66aa72d, f97354f814c54d61a0f39aa2dc145d43; no voters:
I20260807 19:41:59.063934 1504 raft_consensus.cc:2781] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:41:59.064065 1504 raft_consensus.cc:482] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:41:59.064157 1504 raft_consensus.cc:3037] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:59.066391 1504 raft_consensus.cc:504] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:59.066960 1504 leader_election.cc:288] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [CANDIDATE]: Term 1 election: Requested vote from peers ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287), 962ba2e327344703bf1ca14efb11ec49 (127.30.208.3:34923)
I20260807 19:41:59.067516 1427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf8b99067b814c238b7090ba65a45618" candidate_uuid: "f97354f814c54d61a0f39aa2dc145d43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "962ba2e327344703bf1ca14efb11ec49"
I20260807 19:41:59.067497 1280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf8b99067b814c238b7090ba65a45618" candidate_uuid: "f97354f814c54d61a0f39aa2dc145d43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab40e6ac7585412cbf265a8bd66aa72d"
I20260807 19:41:59.067719 1427 raft_consensus.cc:3037] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:59.067737 1280 raft_consensus.cc:3037] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:59.070662 1280 raft_consensus.cc:2454] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f97354f814c54d61a0f39aa2dc145d43 in term 1.
I20260807 19:41:59.070677 1427 raft_consensus.cc:2454] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f97354f814c54d61a0f39aa2dc145d43 in term 1.
I20260807 19:41:59.071095 1053 leader_election.cc:302] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab40e6ac7585412cbf265a8bd66aa72d, f97354f814c54d61a0f39aa2dc145d43; no voters:
I20260807 19:41:59.071406 1504 raft_consensus.cc:2781] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:41:59.071835 1504 raft_consensus.cc:686] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [term 1 LEADER]: Becoming Leader. State: Replica: f97354f814c54d61a0f39aa2dc145d43, State: Running, Role: LEADER
I20260807 19:41:59.072400 1504 consensus_queue.cc:237] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:59.075932 1522 tablet_replica.cc:439] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43: TxnStatusTablet state changed. Reason: New leader f97354f814c54d61a0f39aa2dc145d43. Latest consensus state: current_term: 1 leader_uuid: "f97354f814c54d61a0f39aa2dc145d43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } }
I20260807 19:41:59.076329 1522 tablet_replica.cc:442] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:41:59.077042 1524 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 19:41:59.077404 1524 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 19:41:59.077767 958 catalog_manager.cc:5495] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 reported cstate change: term changed from 0 to 1, leader changed from <none> to f97354f814c54d61a0f39aa2dc145d43 (127.30.208.1). New cstate: current_term: 1 leader_uuid: "f97354f814c54d61a0f39aa2dc145d43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } health_report { overall_health: UNKNOWN } } }
I20260807 19:41:59.132438 1515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:59.126210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:40045 (local address 127.0.0.1:59584)
0807 19:41:59.126453 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:59.126473 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:59.126597 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:59.127087 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:59.127090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:59.127100 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:41:59.127275 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:59.127282 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.128691 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.128695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:59.129763 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:59.129777 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.130744 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.130749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:59.130912 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:59.130975 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:41:59.131995 (+ 1020us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:41:59.132012 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 19:41:59.132067 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 19:41:59.132196 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 19:41:59.132555 1495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:59.126231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59584 (local address 127.30.208.1:40045)
0807 19:41:59.126433 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:59.126440 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:59.126540 (+ 100us) server_negotiation.cc:407] Connection header received
0807 19:41:59.126788 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:59.126794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:59.126905 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:59.127083 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:41:59.127402 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.128579 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.129903 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.130590 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.130719 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:59.131182 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:41:59.131894 (+ 712us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:41:59.132043 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:59.132090 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 19:41:59.132182 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:41:59.141382 1427 raft_consensus.cc:1261] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Refusing update from remote peer f97354f814c54d61a0f39aa2dc145d43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:41:59.141551 1280 raft_consensus.cc:1261] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Refusing update from remote peer f97354f814c54d61a0f39aa2dc145d43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:41:59.142433 1504 consensus_queue.cc:1035] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:59.142763 1522 consensus_queue.cc:1035] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:59.151304 1503 tablet_replica.cc:439] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d: TxnStatusTablet state changed. Reason: New leader f97354f814c54d61a0f39aa2dc145d43. Latest consensus state: current_term: 1 leader_uuid: "f97354f814c54d61a0f39aa2dc145d43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } }
I20260807 19:41:59.151634 1503 tablet_replica.cc:442] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:41:59.153714 1527 mvcc.cc:205] Tried to move back new op lower bound from 7315995521592487936 to 7315995521324703744. Current Snapshot: MvccSnapshot[applied={T|T < 7315995521592487936}]
I20260807 19:41:59.156036 1528 mvcc.cc:205] Tried to move back new op lower bound from 7315995521592487936 to 7315995521324703744. Current Snapshot: MvccSnapshot[applied={T|T < 7315995521592487936}]
I20260807 19:41:59.161773 1522 tablet_replica.cc:439] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f97354f814c54d61a0f39aa2dc145d43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } }
I20260807 19:41:59.162066 1522 tablet_replica.cc:442] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:41:59.182142 1533 tablet_replica.cc:439] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f97354f814c54d61a0f39aa2dc145d43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } }
I20260807 19:41:59.182708 1533 tablet_replica.cc:442] T cf8b99067b814c238b7090ba65a45618 P f97354f814c54d61a0f39aa2dc145d43: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:41:59.201333 1503 tablet_replica.cc:439] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f97354f814c54d61a0f39aa2dc145d43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } }
I20260807 19:41:59.201630 1503 tablet_replica.cc:442] T cf8b99067b814c238b7090ba65a45618 P ab40e6ac7585412cbf265a8bd66aa72d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:41:59.206328 1508 tablet_replica.cc:439] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f97354f814c54d61a0f39aa2dc145d43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } }
I20260807 19:41:59.206707 1508 tablet_replica.cc:442] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:41:59.207008 1508 consensus_queue.cc:1035] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:59.207598 1505 tablet_replica.cc:439] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49: TxnStatusTablet state changed. Reason: New leader f97354f814c54d61a0f39aa2dc145d43. Latest consensus state: current_term: 1 leader_uuid: "f97354f814c54d61a0f39aa2dc145d43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } } }
I20260807 19:41:59.207904 1505 tablet_replica.cc:442] T cf8b99067b814c238b7090ba65a45618 P 962ba2e327344703bf1ca14efb11ec49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:41:59.219018 1490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:59.209799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57524 (local address 127.30.208.3:34923)
0807 19:41:59.209974 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:59.209980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:59.211162 (+ 1182us) server_negotiation.cc:407] Connection header received
0807 19:41:59.211605 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:59.211612 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:59.211728 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:59.211889 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:41:59.212586 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.213750 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.216180 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.216820 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.216948 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:59.217529 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:41:59.218213 (+ 684us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:41:59.218335 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:59.218590 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 19:41:59.218694 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:41:59.221400 1537 consensus_queue.cc:1035] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:59.238617 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:59.233346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:40045 (local address 127.30.208.3:60685)
0807 19:41:59.233928 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:59.233950 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:59.234113 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:59.234444 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:59.234449 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:59.234462 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:41:59.234636 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:59.234644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.235675 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.235681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:59.236691 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:59.236704 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.237495 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.237501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:59.237627 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:59.237685 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:41:59.238076 (+ 391us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:41:59.238088 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:41:59.238160 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 19:41:59.238307 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":335,"threads_started":1}
I20260807 19:41:59.238623 1495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:59.233308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:60685 (local address 127.30.208.1:40045)
0807 19:41:59.233505 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:59.233511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:59.234037 (+ 526us) server_negotiation.cc:407] Connection header received
0807 19:41:59.234241 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:59.234247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:59.234317 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:59.234413 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:41:59.234774 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.235543 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.236832 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.237338 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.237457 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:59.237880 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:41:59.237987 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:41:59.238060 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:59.238197 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 19:41:59.238277 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 19:41:59.277640 1490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:59.270993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:43181 (local address 127.30.208.3:34923)
0807 19:41:59.271157 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:59.271162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:59.271791 (+ 629us) server_negotiation.cc:407] Connection header received
0807 19:41:59.272018 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:59.272024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:59.272115 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:59.272206 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:41:59.273050 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.273995 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.275615 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.276153 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.276269 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:59.276847 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:41:59.276985 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:41:59.277073 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:59.277249 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 19:41:59.277338 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 19:41:59.278044 1546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:59.271044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:34923 (local address 127.30.208.1:43181)
0807 19:41:59.271683 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:59.271706 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:59.271847 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:59.272233 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:59.272238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:59.272251 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:41:59.272687 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:59.272703 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.274120 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.274126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:59.275470 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:59.275484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.276367 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.276372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:59.276543 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:59.276625 (+ 82us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:41:59.277105 (+ 480us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:41:59.277117 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 19:41:59.277225 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 19:41:59.277394 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":320,"threads_started":1}
I20260807 19:41:59.327447 958 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:41:59.327782 958 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:41:59.331802 958 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e: Sending DeleteTablet for 3 replicas of tablet fb43c8f5e5544deb9eb42451e5dcd3d1
I20260807 19:41:59.332950 1401 tablet_service.cc:1514] Processing DeleteTablet for tablet fb43c8f5e5544deb9eb42451e5dcd3d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:59 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:57518
I20260807 19:41:59.333817 1254 tablet_service.cc:1514] Processing DeleteTablet for tablet fb43c8f5e5544deb9eb42451e5dcd3d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:59 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:47006
I20260807 19:41:59.334677 1106 tablet_service.cc:1514] Processing DeleteTablet for tablet fb43c8f5e5544deb9eb42451e5dcd3d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:41:59 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:59570
I20260807 19:41:59.335992 958 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:41:59.337445 958 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:41:59.348305 1550 tablet_replica.cc:330] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d: stopping tablet replica
I20260807 19:41:59.349005 1550 raft_consensus.cc:2229] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:41:59.349496 1550 raft_consensus.cc:2258] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:59.351464 1550 ts_tablet_manager.cc:1905] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:59.353410 1400 tablet_service.cc:1467] Processing CreateTablet for tablet 898bc1798244469b8d660d09a08838ac (DEFAULT_TABLE table=test-table [id=58889add7bf84a90809a062faa0ab2f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:59.353497 1254 tablet_service.cc:1467] Processing CreateTablet for tablet 898bc1798244469b8d660d09a08838ac (DEFAULT_TABLE table=test-table [id=58889add7bf84a90809a062faa0ab2f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:59.353950 1254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898bc1798244469b8d660d09a08838ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:59.354001 1400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898bc1798244469b8d660d09a08838ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:59.358249 1105 tablet_service.cc:1467] Processing CreateTablet for tablet 898bc1798244469b8d660d09a08838ac (DEFAULT_TABLE table=test-table [id=58889add7bf84a90809a062faa0ab2f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:41:59.358865 1105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898bc1798244469b8d660d09a08838ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:41:59.365659 1552 tablet_bootstrap.cc:492] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d: Bootstrap starting.
I20260807 19:41:59.367635 1552 tablet_bootstrap.cc:654] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:59.383023 1554 tablet_bootstrap.cc:492] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43: Bootstrap starting.
I20260807 19:41:59.384598 1549 tablet_replica.cc:330] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49: stopping tablet replica
I20260807 19:41:59.385221 1549 raft_consensus.cc:2229] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:41:59.386056 1549 raft_consensus.cc:2258] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:59.383911 1551 tablet_replica.cc:330] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43: stopping tablet replica
I20260807 19:41:59.386938 1551 raft_consensus.cc:2229] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:41:59.387341 1551 raft_consensus.cc:2258] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:41:59.387746 1553 tablet_bootstrap.cc:492] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49: Bootstrap starting.
I20260807 19:41:59.390093 1554 tablet_bootstrap.cc:654] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:59.390479 1552 tablet_bootstrap.cc:492] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d: No bootstrap required, opened a new log
I20260807 19:41:59.390648 1552 ts_tablet_manager.cc:1397] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d: Time spent bootstrapping tablet: real 0.025s user 0.008s sys 0.000s
I20260807 19:41:59.391474 1552 raft_consensus.cc:348] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:59.391723 1552 raft_consensus.cc:374] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:59.391819 1552 raft_consensus.cc:729] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab40e6ac7585412cbf265a8bd66aa72d, State: Initialized, Role: FOLLOWER
I20260807 19:41:59.391987 1549 ts_tablet_manager.cc:1905] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:59.392079 1552 consensus_queue.cc:260] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:59.392683 1552 ts_tablet_manager.cc:1428] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 19:41:59.401755 1549 ts_tablet_manager.cc:1918] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:41:59.402002 1549 log.cc:1196] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/fb43c8f5e5544deb9eb42451e5dcd3d1
I20260807 19:41:59.404572 1550 ts_tablet_manager.cc:1918] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:41:59.404758 1550 log.cc:1196] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/fb43c8f5e5544deb9eb42451e5dcd3d1
I20260807 19:41:59.405280 1550 ts_tablet_manager.cc:1939] T fb43c8f5e5544deb9eb42451e5dcd3d1 P ab40e6ac7585412cbf265a8bd66aa72d: Deleting consensus metadata
I20260807 19:41:59.397929 1551 ts_tablet_manager.cc:1905] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:41:59.414374 1553 tablet_bootstrap.cc:654] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49: Neither blocks nor log segments found. Creating new log.
I20260807 19:41:59.414469 1549 ts_tablet_manager.cc:1939] T fb43c8f5e5544deb9eb42451e5dcd3d1 P 962ba2e327344703bf1ca14efb11ec49: Deleting consensus metadata
I20260807 19:41:59.416548 946 catalog_manager.cc:4841] TS 962ba2e327344703bf1ca14efb11ec49 (127.30.208.3:34923): tablet fb43c8f5e5544deb9eb42451e5dcd3d1 (table test-table [id=ab718851c7e44d1fa70479aa8e1a8a61]) successfully deleted
I20260807 19:41:59.419864 1551 ts_tablet_manager.cc:1918] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:41:59.420413 945 catalog_manager.cc:4841] TS ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287): tablet fb43c8f5e5544deb9eb42451e5dcd3d1 (table test-table [id=ab718851c7e44d1fa70479aa8e1a8a61]) successfully deleted
I20260807 19:41:59.422683 1551 log.cc:1196] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/fb43c8f5e5544deb9eb42451e5dcd3d1
I20260807 19:41:59.423437 1551 ts_tablet_manager.cc:1939] T fb43c8f5e5544deb9eb42451e5dcd3d1 P f97354f814c54d61a0f39aa2dc145d43: Deleting consensus metadata
I20260807 19:41:59.425536 946 catalog_manager.cc:4841] TS f97354f814c54d61a0f39aa2dc145d43 (127.30.208.1:40045): tablet fb43c8f5e5544deb9eb42451e5dcd3d1 (table test-table [id=ab718851c7e44d1fa70479aa8e1a8a61]) successfully deleted
I20260807 19:41:59.427605 1553 tablet_bootstrap.cc:492] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49: No bootstrap required, opened a new log
I20260807 19:41:59.427781 1553 ts_tablet_manager.cc:1397] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.009s
I20260807 19:41:59.428700 1553 raft_consensus.cc:348] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:59.428985 1553 raft_consensus.cc:374] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:59.429141 1553 raft_consensus.cc:729] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 962ba2e327344703bf1ca14efb11ec49, State: Initialized, Role: FOLLOWER
I20260807 19:41:59.429571 1553 consensus_queue.cc:260] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:59.430272 1553 ts_tablet_manager.cc:1428] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:41:59.431826 1554 tablet_bootstrap.cc:492] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43: No bootstrap required, opened a new log
I20260807 19:41:59.432044 1554 ts_tablet_manager.cc:1397] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43: Time spent bootstrapping tablet: real 0.049s user 0.006s sys 0.000s
I20260807 19:41:59.432968 1554 raft_consensus.cc:348] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:59.433331 1554 raft_consensus.cc:374] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:41:59.433492 1554 raft_consensus.cc:729] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97354f814c54d61a0f39aa2dc145d43, State: Initialized, Role: FOLLOWER
I20260807 19:41:59.433837 1554 consensus_queue.cc:260] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:59.437078 1554 ts_tablet_manager.cc:1428] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260807 19:41:59.806348 1533 raft_consensus.cc:482] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:41:59.806602 1533 raft_consensus.cc:504] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:59.807404 1533 leader_election.cc:288] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287), 962ba2e327344703bf1ca14efb11ec49 (127.30.208.3:34923)
I20260807 19:41:59.808121 1280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898bc1798244469b8d660d09a08838ac" candidate_uuid: "f97354f814c54d61a0f39aa2dc145d43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" is_pre_election: true
I20260807 19:41:59.808121 1427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898bc1798244469b8d660d09a08838ac" candidate_uuid: "f97354f814c54d61a0f39aa2dc145d43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "962ba2e327344703bf1ca14efb11ec49" is_pre_election: true
I20260807 19:41:59.808414 1427 raft_consensus.cc:2454] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f97354f814c54d61a0f39aa2dc145d43 in term 0.
I20260807 19:41:59.808420 1280 raft_consensus.cc:2454] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f97354f814c54d61a0f39aa2dc145d43 in term 0.
I20260807 19:41:59.808863 1053 leader_election.cc:302] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 962ba2e327344703bf1ca14efb11ec49, ab40e6ac7585412cbf265a8bd66aa72d, f97354f814c54d61a0f39aa2dc145d43; no voters:
I20260807 19:41:59.809150 1533 raft_consensus.cc:2781] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:41:59.809296 1533 raft_consensus.cc:482] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:41:59.809409 1533 raft_consensus.cc:3037] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:59.811584 1533 raft_consensus.cc:504] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:59.812166 1533 leader_election.cc:288] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [CANDIDATE]: Term 1 election: Requested vote from peers ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287), 962ba2e327344703bf1ca14efb11ec49 (127.30.208.3:34923)
I20260807 19:41:59.812721 1427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898bc1798244469b8d660d09a08838ac" candidate_uuid: "f97354f814c54d61a0f39aa2dc145d43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "962ba2e327344703bf1ca14efb11ec49"
I20260807 19:41:59.812929 1427 raft_consensus.cc:3037] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:59.813089 1280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898bc1798244469b8d660d09a08838ac" candidate_uuid: "f97354f814c54d61a0f39aa2dc145d43" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab40e6ac7585412cbf265a8bd66aa72d"
I20260807 19:41:59.813387 1280 raft_consensus.cc:3037] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:41:59.815820 1427 raft_consensus.cc:2454] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f97354f814c54d61a0f39aa2dc145d43 in term 1.
I20260807 19:41:59.816166 1280 raft_consensus.cc:2454] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f97354f814c54d61a0f39aa2dc145d43 in term 1.
I20260807 19:41:59.816347 1054 leader_election.cc:302] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 962ba2e327344703bf1ca14efb11ec49, f97354f814c54d61a0f39aa2dc145d43; no voters:
I20260807 19:41:59.816681 1533 raft_consensus.cc:2781] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:41:59.817040 1533 raft_consensus.cc:686] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [term 1 LEADER]: Becoming Leader. State: Replica: f97354f814c54d61a0f39aa2dc145d43, State: Running, Role: LEADER
I20260807 19:41:59.817349 1533 consensus_queue.cc:237] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:41:59.819927 959 catalog_manager.cc:5495] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 reported cstate change: term changed from 0 to 1, leader changed from <none> to f97354f814c54d61a0f39aa2dc145d43 (127.30.208.1). New cstate: current_term: 1 leader_uuid: "f97354f814c54d61a0f39aa2dc145d43" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } health_report { overall_health: UNKNOWN } } }
I20260807 19:41:59.846030 1569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:59.841232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37942 (local address 127.30.208.1:40045)
0807 19:41:59.841722 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:59.841727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:59.841788 (+ 61us) server_negotiation.cc:407] Connection header received
0807 19:41:59.841845 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:59.841848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:59.841910 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:59.841993 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:41:59.842352 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.843079 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.844324 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.844730 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.844831 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:59.845324 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:41:59.845426 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:41:59.845490 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:59.845606 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 19:41:59.845677 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":255,"threads_started":1}
I20260807 19:41:59.860277 1569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:59.854794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:36591 (local address 127.30.208.1:40045)
0807 19:41:59.854959 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 19:41:59.854963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:41:59.855355 (+ 392us) server_negotiation.cc:407] Connection header received
0807 19:41:59.855492 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:41:59.855497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:41:59.855576 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:41:59.855662 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:41:59.856060 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.857248 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.858551 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.859160 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.859264 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:59.859591 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:41:59.859709 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:41:59.859780 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 19:41:59.859867 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 19:41:59.859965 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":36}
I20260807 19:41:59.860311 1574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:41:59.854620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:40045 (local address 127.30.208.1:36591)
0807 19:41:59.855263 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0807 19:41:59.855277 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:41:59.855392 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:41:59.855668 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:41:59.855670 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:41:59.855678 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:41:59.855937 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:59.855945 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.857374 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.857378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:59.858425 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:41:59.858438 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:41:59.859314 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:41:59.859318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:41:59.859428 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:41:59.859461 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:41:59.859787 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:41:59.859793 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:41:59.859847 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 19:41:59.859981 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":3,"thread_start_us":403,"threads_started":1}
I20260807 19:41:59.868214 1427 raft_consensus.cc:1261] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Refusing update from remote peer f97354f814c54d61a0f39aa2dc145d43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:41:59.868240 1279 raft_consensus.cc:1261] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Refusing update from remote peer f97354f814c54d61a0f39aa2dc145d43: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:41:59.868862 1558 consensus_queue.cc:1035] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:59.869127 1533 consensus_queue.cc:1035] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:41:59.873806 1562 mvcc.cc:205] Tried to move back new op lower bound from 7315995524574625792 to 7315995524374052864. Current Snapshot: MvccSnapshot[applied={T|T < 7315995524574625792}]
W20260807 19:41:59.889751 1560 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 19:42:01.484613 959 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:01.484853 959 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:01.489148 959 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e: Sending DeleteTablet for 3 replicas of tablet 898bc1798244469b8d660d09a08838ac
I20260807 19:42:01.490046 1105 tablet_service.cc:1514] Processing DeleteTablet for tablet 898bc1798244469b8d660d09a08838ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:01 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:59570
I20260807 19:42:01.490162 1254 tablet_service.cc:1514] Processing DeleteTablet for tablet 898bc1798244469b8d660d09a08838ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:01 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:47006
I20260807 19:42:01.490200 1400 tablet_service.cc:1514] Processing DeleteTablet for tablet 898bc1798244469b8d660d09a08838ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:01 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:57518
I20260807 19:42:01.502955 1603 tablet_replica.cc:330] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49: stopping tablet replica
I20260807 19:42:01.503222 1603 raft_consensus.cc:2229] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:01.503525 1603 raft_consensus.cc:2258] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:01.504833 1603 ts_tablet_manager.cc:1905] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:01.505764 1602 tablet_replica.cc:330] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d: stopping tablet replica
I20260807 19:42:01.506062 1602 raft_consensus.cc:2229] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:01.506450 1602 raft_consensus.cc:2258] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:01.506577 1601 tablet_replica.cc:330] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43: stopping tablet replica
I20260807 19:42:01.506775 1601 raft_consensus.cc:2229] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:01.507498 1602 ts_tablet_manager.cc:1905] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:01.508682 1601 raft_consensus.cc:2258] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:01.509670 1601 ts_tablet_manager.cc:1905] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 19:42:01 dist-test-slave-vq2q krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131721, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260807 19:42:01.529016 1603 ts_tablet_manager.cc:1918] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:42:01.529467 1602 ts_tablet_manager.cc:1918] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:42:01.529676 1602 log.cc:1196] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/898bc1798244469b8d660d09a08838ac
I20260807 19:42:01.529919 1603 log.cc:1196] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/898bc1798244469b8d660d09a08838ac
I20260807 19:42:01.530103 1602 ts_tablet_manager.cc:1939] T 898bc1798244469b8d660d09a08838ac P ab40e6ac7585412cbf265a8bd66aa72d: Deleting consensus metadata
I20260807 19:42:01.530563 1603 ts_tablet_manager.cc:1939] T 898bc1798244469b8d660d09a08838ac P 962ba2e327344703bf1ca14efb11ec49: Deleting consensus metadata
I20260807 19:42:01.581328 946 catalog_manager.cc:4841] TS 962ba2e327344703bf1ca14efb11ec49 (127.30.208.3:34923): tablet 898bc1798244469b8d660d09a08838ac (table test-table [id=58889add7bf84a90809a062faa0ab2f6]) successfully deleted
I20260807 19:42:01.583364 945 catalog_manager.cc:4841] TS ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287): tablet 898bc1798244469b8d660d09a08838ac (table test-table [id=58889add7bf84a90809a062faa0ab2f6]) successfully deleted
W20260807 19:42:01.626590 31552 mini_kdc.cc:317] Time spent kinit for test-user: real 0.135s user 0.003s sys 0.011s
I20260807 19:42:01.626760 1601 ts_tablet_manager.cc:1918] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:42:01.626945 1601 log.cc:1196] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/898bc1798244469b8d660d09a08838ac
I20260807 19:42:01.627497 1601 ts_tablet_manager.cc:1939] T 898bc1798244469b8d660d09a08838ac P f97354f814c54d61a0f39aa2dc145d43: Deleting consensus metadata
I20260807 19:42:01.628952 946 catalog_manager.cc:4841] TS f97354f814c54d61a0f39aa2dc145d43 (127.30.208.1:40045): tablet 898bc1798244469b8d660d09a08838ac (table test-table [id=58889add7bf84a90809a062faa0ab2f6]) successfully deleted
Aug 07 19:42:01 dist-test-slave-vq2q krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131721, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:01.644678 1611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:01.630185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38502 (local address 127.30.208.62:33679)
0807 19:42:01.631281 (+ 1096us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:01.631285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:01.631307 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:42:01.631360 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:01.631364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:01.631423 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:01.631514 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:01.632940 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:01.633803 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:01.635128 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:01.635580 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:01.635684 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:01.639590 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:01.639596 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:01.639599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:01.639640 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:01.642311 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:01.643018 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:01.643022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:01.643024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:01.643153 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:01.643452 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:01.643455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:01.643456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:01.643720 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:01.643888 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:01.644216 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 19:42:01.644407 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":978,"thread_start_us":826,"threads_started":1}
I20260807 19:42:01.648341 959 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:01.648960 959 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:01.655475 1105 tablet_service.cc:1467] Processing CreateTablet for tablet 93594a0454cf4296a6f0f402e22f0619 (DEFAULT_TABLE table=test-table [id=019917780d0c41099d429b84376dead5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:01.655609 1254 tablet_service.cc:1467] Processing CreateTablet for tablet 93594a0454cf4296a6f0f402e22f0619 (DEFAULT_TABLE table=test-table [id=019917780d0c41099d429b84376dead5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:01.655766 1400 tablet_service.cc:1467] Processing CreateTablet for tablet 93594a0454cf4296a6f0f402e22f0619 (DEFAULT_TABLE table=test-table [id=019917780d0c41099d429b84376dead5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:01.656000 1254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93594a0454cf4296a6f0f402e22f0619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:01.655987 1105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93594a0454cf4296a6f0f402e22f0619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:01.656170 1400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93594a0454cf4296a6f0f402e22f0619. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:01.661792 1613 tablet_bootstrap.cc:492] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d: Bootstrap starting.
I20260807 19:42:01.663362 1613 tablet_bootstrap.cc:654] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:01.664927 1615 tablet_bootstrap.cc:492] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43: Bootstrap starting.
I20260807 19:42:01.665625 1613 tablet_bootstrap.cc:492] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d: No bootstrap required, opened a new log
I20260807 19:42:01.665721 1614 tablet_bootstrap.cc:492] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49: Bootstrap starting.
I20260807 19:42:01.666253 1613 ts_tablet_manager.cc:1397] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260807 19:42:01.666723 1615 tablet_bootstrap.cc:654] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:01.667269 1614 tablet_bootstrap.cc:654] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:01.667609 1613 raft_consensus.cc:348] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:01.667920 1613 raft_consensus.cc:374] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:01.668013 1613 raft_consensus.cc:729] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab40e6ac7585412cbf265a8bd66aa72d, State: Initialized, Role: FOLLOWER
I20260807 19:42:01.668349 1613 consensus_queue.cc:260] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:01.668884 1615 tablet_bootstrap.cc:492] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43: No bootstrap required, opened a new log
I20260807 19:42:01.669016 1615 ts_tablet_manager.cc:1397] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 19:42:01.669248 1613 ts_tablet_manager.cc:1428] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:42:01.669724 1614 tablet_bootstrap.cc:492] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49: No bootstrap required, opened a new log
I20260807 19:42:01.669874 1614 ts_tablet_manager.cc:1397] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 19:42:01.669813 1615 raft_consensus.cc:348] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:01.670069 1615 raft_consensus.cc:374] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:01.670161 1615 raft_consensus.cc:729] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97354f814c54d61a0f39aa2dc145d43, State: Initialized, Role: FOLLOWER
I20260807 19:42:01.670397 1615 consensus_queue.cc:260] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:01.670640 1614 raft_consensus.cc:348] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:01.670912 1614 raft_consensus.cc:374] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:01.670922 1615 ts_tablet_manager.cc:1428] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 19:42:01.671017 1614 raft_consensus.cc:729] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 962ba2e327344703bf1ca14efb11ec49, State: Initialized, Role: FOLLOWER
I20260807 19:42:01.671283 1614 consensus_queue.cc:260] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:01.673708 1614 ts_tablet_manager.cc:1428] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260807 19:42:01.727411 1620 raft_consensus.cc:482] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:01.727684 1620 raft_consensus.cc:504] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:01.728574 1620 leader_election.cc:288] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f97354f814c54d61a0f39aa2dc145d43 (127.30.208.1:40045), ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287)
I20260807 19:42:01.729226 1127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93594a0454cf4296a6f0f402e22f0619" candidate_uuid: "962ba2e327344703bf1ca14efb11ec49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f97354f814c54d61a0f39aa2dc145d43" is_pre_election: true
I20260807 19:42:01.729288 1280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93594a0454cf4296a6f0f402e22f0619" candidate_uuid: "962ba2e327344703bf1ca14efb11ec49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" is_pre_election: true
I20260807 19:42:01.729493 1127 raft_consensus.cc:2454] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 962ba2e327344703bf1ca14efb11ec49 in term 0.
I20260807 19:42:01.729547 1280 raft_consensus.cc:2454] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 962ba2e327344703bf1ca14efb11ec49 in term 0.
I20260807 19:42:01.729997 1350 leader_election.cc:302] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 962ba2e327344703bf1ca14efb11ec49, f97354f814c54d61a0f39aa2dc145d43; no voters:
I20260807 19:42:01.730479 1620 raft_consensus.cc:2781] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:01.730592 1620 raft_consensus.cc:482] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:01.730691 1620 raft_consensus.cc:3037] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:01.828029 1620 raft_consensus.cc:504] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:01.828780 1620 leader_election.cc:288] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [CANDIDATE]: Term 1 election: Requested vote from peers f97354f814c54d61a0f39aa2dc145d43 (127.30.208.1:40045), ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287)
I20260807 19:42:01.829589 1280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93594a0454cf4296a6f0f402e22f0619" candidate_uuid: "962ba2e327344703bf1ca14efb11ec49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab40e6ac7585412cbf265a8bd66aa72d"
I20260807 19:42:01.829825 1280 raft_consensus.cc:3037] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:01.829955 1127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93594a0454cf4296a6f0f402e22f0619" candidate_uuid: "962ba2e327344703bf1ca14efb11ec49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f97354f814c54d61a0f39aa2dc145d43"
I20260807 19:42:01.830209 1127 raft_consensus.cc:3037] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:01.832175 1280 raft_consensus.cc:2454] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 962ba2e327344703bf1ca14efb11ec49 in term 1.
I20260807 19:42:01.832715 1349 leader_election.cc:302] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 962ba2e327344703bf1ca14efb11ec49, ab40e6ac7585412cbf265a8bd66aa72d; no voters:
I20260807 19:42:01.833048 1620 raft_consensus.cc:2781] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:01.833288 1127 raft_consensus.cc:2454] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 962ba2e327344703bf1ca14efb11ec49 in term 1.
I20260807 19:42:01.833622 1620 raft_consensus.cc:686] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [term 1 LEADER]: Becoming Leader. State: Replica: 962ba2e327344703bf1ca14efb11ec49, State: Running, Role: LEADER
I20260807 19:42:01.833951 1620 consensus_queue.cc:237] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:01.836642 959 catalog_manager.cc:5495] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 962ba2e327344703bf1ca14efb11ec49 (127.30.208.3). New cstate: current_term: 1 leader_uuid: "962ba2e327344703bf1ca14efb11ec49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } health_report { overall_health: HEALTHY } } }
I20260807 19:42:01.874325 1625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:01.869335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36408 (local address 127.30.208.3:34923)
0807 19:42:01.869965 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:01.869970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:01.869991 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:42:01.870061 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:01.870066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:01.870135 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:01.870228 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:01.870563 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:01.871331 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:01.872692 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:01.873146 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:01.873278 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:01.873666 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:01.873775 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:01.873840 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:01.873938 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 19:42:01.874001 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":356,"threads_started":1}
I20260807 19:42:01.883764 1280 raft_consensus.cc:1261] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Refusing update from remote peer 962ba2e327344703bf1ca14efb11ec49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:01.883849 1127 raft_consensus.cc:1261] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Refusing update from remote peer 962ba2e327344703bf1ca14efb11ec49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:01.884287 1621 consensus_queue.cc:1035] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:01.884653 1620 consensus_queue.cc:1035] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:01.935835 959 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:01.936069 959 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:01.939000 959 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e: Sending DeleteTablet for 3 replicas of tablet 93594a0454cf4296a6f0f402e22f0619
I20260807 19:42:01.939837 1105 tablet_service.cc:1514] Processing DeleteTablet for tablet 93594a0454cf4296a6f0f402e22f0619 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:01 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:59570
I20260807 19:42:01.939895 1400 tablet_service.cc:1514] Processing DeleteTablet for tablet 93594a0454cf4296a6f0f402e22f0619 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:01 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:57518
I20260807 19:42:01.939944 1254 tablet_service.cc:1514] Processing DeleteTablet for tablet 93594a0454cf4296a6f0f402e22f0619 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:01 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:47006
I20260807 19:42:01.940156 1601 tablet_replica.cc:330] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43: stopping tablet replica
I20260807 19:42:01.940181 1603 tablet_replica.cc:330] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49: stopping tablet replica
I20260807 19:42:01.940201 1602 tablet_replica.cc:330] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d: stopping tablet replica
I20260807 19:42:01.940371 1601 raft_consensus.cc:2229] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:01.940378 1602 raft_consensus.cc:2229] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:01.940385 1603 raft_consensus.cc:2229] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:01.940579 1602 raft_consensus.cc:2258] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:01.940579 1601 raft_consensus.cc:2258] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:01.940726 1603 raft_consensus.cc:2258] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:01.941004 959 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38502:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:01.941740 959 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:01.941836 1603 ts_tablet_manager.cc:1905] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:01.941846 1601 ts_tablet_manager.cc:1905] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:01.942124 1602 ts_tablet_manager.cc:1905] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:01.949667 1105 tablet_service.cc:1467] Processing CreateTablet for tablet c01ce0292b5a4d6eb22e53faef90d47c (DEFAULT_TABLE table=test-table [id=7657fd003263427b8373b080452d6316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:01.949893 1400 tablet_service.cc:1467] Processing CreateTablet for tablet c01ce0292b5a4d6eb22e53faef90d47c (DEFAULT_TABLE table=test-table [id=7657fd003263427b8373b080452d6316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:01.949869 1254 tablet_service.cc:1467] Processing CreateTablet for tablet c01ce0292b5a4d6eb22e53faef90d47c (DEFAULT_TABLE table=test-table [id=7657fd003263427b8373b080452d6316]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:02.032397 1105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c01ce0292b5a4d6eb22e53faef90d47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:02.032970 1400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c01ce0292b5a4d6eb22e53faef90d47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:02.036448 1601 ts_tablet_manager.cc:1918] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:42:02.036705 1601 log.cc:1196] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/93594a0454cf4296a6f0f402e22f0619
I20260807 19:42:02.037274 1601 ts_tablet_manager.cc:1939] T 93594a0454cf4296a6f0f402e22f0619 P f97354f814c54d61a0f39aa2dc145d43: Deleting consensus metadata
I20260807 19:42:02.037890 1254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c01ce0292b5a4d6eb22e53faef90d47c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:02.041141 1615 tablet_bootstrap.cc:492] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43: Bootstrap starting.
I20260807 19:42:02.042840 1615 tablet_bootstrap.cc:654] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:02.044216 1614 tablet_bootstrap.cc:492] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49: Bootstrap starting.
I20260807 19:42:02.044490 1613 tablet_bootstrap.cc:492] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d: Bootstrap starting.
I20260807 19:42:02.045894 1614 tablet_bootstrap.cc:654] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:02.045933 1615 tablet_bootstrap.cc:492] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43: No bootstrap required, opened a new log
I20260807 19:42:02.046067 1613 tablet_bootstrap.cc:654] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:02.046078 1615 ts_tablet_manager.cc:1397] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 19:42:02.046777 1615 raft_consensus.cc:348] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:02.047036 1615 raft_consensus.cc:374] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:02.047118 1615 raft_consensus.cc:729] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97354f814c54d61a0f39aa2dc145d43, State: Initialized, Role: FOLLOWER
I20260807 19:42:02.047408 1615 consensus_queue.cc:260] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:02.048038 1615 ts_tablet_manager.cc:1428] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:42:02.048159 1613 tablet_bootstrap.cc:492] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d: No bootstrap required, opened a new log
I20260807 19:42:02.048161 1614 tablet_bootstrap.cc:492] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49: No bootstrap required, opened a new log
I20260807 19:42:02.048329 1614 ts_tablet_manager.cc:1397] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260807 19:42:02.048329 1613 ts_tablet_manager.cc:1397] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.002s
I20260807 19:42:02.049024 1614 raft_consensus.cc:348] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:02.049026 1613 raft_consensus.cc:348] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:02.049302 1614 raft_consensus.cc:374] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:02.049302 1613 raft_consensus.cc:374] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:02.049386 1614 raft_consensus.cc:729] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 962ba2e327344703bf1ca14efb11ec49, State: Initialized, Role: FOLLOWER
I20260807 19:42:02.049396 1613 raft_consensus.cc:729] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab40e6ac7585412cbf265a8bd66aa72d, State: Initialized, Role: FOLLOWER
I20260807 19:42:02.049619 1613 consensus_queue.cc:260] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:02.049646 1614 consensus_queue.cc:260] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:02.050221 1613 ts_tablet_manager.cc:1428] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:42:02.050292 1614 ts_tablet_manager.cc:1428] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260807 19:42:02.111321 946 catalog_manager.cc:4841] TS f97354f814c54d61a0f39aa2dc145d43 (127.30.208.1:40045): tablet 93594a0454cf4296a6f0f402e22f0619 (table test-table [id=019917780d0c41099d429b84376dead5]) successfully deleted
I20260807 19:42:02.133944 1602 ts_tablet_manager.cc:1918] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:42:02.133944 1603 ts_tablet_manager.cc:1918] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:42:02.134151 1603 log.cc:1196] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/93594a0454cf4296a6f0f402e22f0619
I20260807 19:42:02.134157 1602 log.cc:1196] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/93594a0454cf4296a6f0f402e22f0619
I20260807 19:42:02.134541 1602 ts_tablet_manager.cc:1939] T 93594a0454cf4296a6f0f402e22f0619 P ab40e6ac7585412cbf265a8bd66aa72d: Deleting consensus metadata
I20260807 19:42:02.134654 1603 ts_tablet_manager.cc:1939] T 93594a0454cf4296a6f0f402e22f0619 P 962ba2e327344703bf1ca14efb11ec49: Deleting consensus metadata
I20260807 19:42:02.136288 946 catalog_manager.cc:4841] TS 962ba2e327344703bf1ca14efb11ec49 (127.30.208.3:34923): tablet 93594a0454cf4296a6f0f402e22f0619 (table test-table [id=019917780d0c41099d429b84376dead5]) successfully deleted
I20260807 19:42:02.136301 945 catalog_manager.cc:4841] TS ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287): tablet 93594a0454cf4296a6f0f402e22f0619 (table test-table [id=019917780d0c41099d429b84376dead5]) successfully deleted
I20260807 19:42:02.258836 1621 raft_consensus.cc:482] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:02.259092 1621 raft_consensus.cc:504] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:02.259847 1621 leader_election.cc:288] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f97354f814c54d61a0f39aa2dc145d43 (127.30.208.1:40045), ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287)
I20260807 19:42:02.260452 1127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c01ce0292b5a4d6eb22e53faef90d47c" candidate_uuid: "962ba2e327344703bf1ca14efb11ec49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f97354f814c54d61a0f39aa2dc145d43" is_pre_election: true
I20260807 19:42:02.260514 1280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c01ce0292b5a4d6eb22e53faef90d47c" candidate_uuid: "962ba2e327344703bf1ca14efb11ec49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" is_pre_election: true
I20260807 19:42:02.260751 1280 raft_consensus.cc:2454] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 962ba2e327344703bf1ca14efb11ec49 in term 0.
I20260807 19:42:02.260772 1127 raft_consensus.cc:2454] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 962ba2e327344703bf1ca14efb11ec49 in term 0.
I20260807 19:42:02.261158 1349 leader_election.cc:302] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 962ba2e327344703bf1ca14efb11ec49, ab40e6ac7585412cbf265a8bd66aa72d; no voters:
I20260807 19:42:02.261466 1621 raft_consensus.cc:2781] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:02.261574 1621 raft_consensus.cc:482] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:02.261682 1621 raft_consensus.cc:3037] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:02.263707 1621 raft_consensus.cc:504] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:02.264278 1621 leader_election.cc:288] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [CANDIDATE]: Term 1 election: Requested vote from peers f97354f814c54d61a0f39aa2dc145d43 (127.30.208.1:40045), ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287)
I20260807 19:42:02.264708 1127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c01ce0292b5a4d6eb22e53faef90d47c" candidate_uuid: "962ba2e327344703bf1ca14efb11ec49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f97354f814c54d61a0f39aa2dc145d43"
I20260807 19:42:02.264729 1280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c01ce0292b5a4d6eb22e53faef90d47c" candidate_uuid: "962ba2e327344703bf1ca14efb11ec49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab40e6ac7585412cbf265a8bd66aa72d"
I20260807 19:42:02.264899 1280 raft_consensus.cc:3037] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:02.264923 1127 raft_consensus.cc:3037] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:02.266906 1127 raft_consensus.cc:2454] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 962ba2e327344703bf1ca14efb11ec49 in term 1.
I20260807 19:42:02.266916 1280 raft_consensus.cc:2454] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 962ba2e327344703bf1ca14efb11ec49 in term 1.
I20260807 19:42:02.267310 1349 leader_election.cc:302] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 962ba2e327344703bf1ca14efb11ec49, ab40e6ac7585412cbf265a8bd66aa72d; no voters:
I20260807 19:42:02.267553 1621 raft_consensus.cc:2781] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:02.267726 1621 raft_consensus.cc:686] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [term 1 LEADER]: Becoming Leader. State: Replica: 962ba2e327344703bf1ca14efb11ec49, State: Running, Role: LEADER
I20260807 19:42:02.267962 1621 consensus_queue.cc:237] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } }
I20260807 19:42:02.270406 959 catalog_manager.cc:5495] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 962ba2e327344703bf1ca14efb11ec49 (127.30.208.3). New cstate: current_term: 1 leader_uuid: "962ba2e327344703bf1ca14efb11ec49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "962ba2e327344703bf1ca14efb11ec49" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34923 } health_report { overall_health: HEALTHY } } }
I20260807 19:42:02.333581 1280 raft_consensus.cc:1261] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Refusing update from remote peer 962ba2e327344703bf1ca14efb11ec49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:02.333647 1127 raft_consensus.cc:1261] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Refusing update from remote peer 962ba2e327344703bf1ca14efb11ec49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:02.334192 1620 consensus_queue.cc:1035] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab40e6ac7585412cbf265a8bd66aa72d" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 41287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:02.334462 1621 consensus_queue.cc:1035] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f97354f814c54d61a0f39aa2dc145d43" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:42:02.352226 1584 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 19:42:03.934507 958 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:03.934725 958 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38502:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:03.938947 958 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fae44f708a8f4556b4231a114013300e: Sending DeleteTablet for 3 replicas of tablet c01ce0292b5a4d6eb22e53faef90d47c
I20260807 19:42:03.939821 1105 tablet_service.cc:1514] Processing DeleteTablet for tablet c01ce0292b5a4d6eb22e53faef90d47c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:03 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:59570
I20260807 19:42:03.939833 1254 tablet_service.cc:1514] Processing DeleteTablet for tablet c01ce0292b5a4d6eb22e53faef90d47c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:03 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:47006
I20260807 19:42:03.939918 1400 tablet_service.cc:1514] Processing DeleteTablet for tablet c01ce0292b5a4d6eb22e53faef90d47c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:03 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:57518
I20260807 19:42:03.940837 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 1034
I20260807 19:42:03.952278 1669 tablet_replica.cc:330] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49: stopping tablet replica
I20260807 19:42:03.952862 1668 tablet_replica.cc:330] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d: stopping tablet replica
I20260807 19:42:03.953086 1667 tablet_replica.cc:330] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43: stopping tablet replica
I20260807 19:42:03.953080 1669 raft_consensus.cc:2229] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:03.953246 1668 raft_consensus.cc:2229] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:03.953339 1667 raft_consensus.cc:2229] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:03.953531 1669 raft_consensus.cc:2258] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:03.953606 1667 raft_consensus.cc:2258] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:03.953593 1668 raft_consensus.cc:2258] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:03.954473 1669 ts_tablet_manager.cc:1905] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:03.954627 1667 ts_tablet_manager.cc:1905] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:03.955117 1668 ts_tablet_manager.cc:1905] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:03.961387 1667 ts_tablet_manager.cc:1918] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:42:03.961390 1669 ts_tablet_manager.cc:1918] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:42:03.961539 1669 log.cc:1196] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/c01ce0292b5a4d6eb22e53faef90d47c
I20260807 19:42:03.961542 1667 log.cc:1196] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/c01ce0292b5a4d6eb22e53faef90d47c
I20260807 19:42:03.961807 1668 ts_tablet_manager.cc:1918] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 19:42:03.961959 1668 log.cc:1196] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/c01ce0292b5a4d6eb22e53faef90d47c
I20260807 19:42:03.961971 1669 ts_tablet_manager.cc:1939] T c01ce0292b5a4d6eb22e53faef90d47c P 962ba2e327344703bf1ca14efb11ec49: Deleting consensus metadata
I20260807 19:42:03.962080 1667 ts_tablet_manager.cc:1939] T c01ce0292b5a4d6eb22e53faef90d47c P f97354f814c54d61a0f39aa2dc145d43: Deleting consensus metadata
I20260807 19:42:03.962354 1668 ts_tablet_manager.cc:1939] T c01ce0292b5a4d6eb22e53faef90d47c P ab40e6ac7585412cbf265a8bd66aa72d: Deleting consensus metadata
I20260807 19:42:03.963114 946 catalog_manager.cc:4841] TS 962ba2e327344703bf1ca14efb11ec49 (127.30.208.3:34923): tablet c01ce0292b5a4d6eb22e53faef90d47c (table test-table [id=7657fd003263427b8373b080452d6316]) successfully deleted
I20260807 19:42:03.963390 946 catalog_manager.cc:4841] TS f97354f814c54d61a0f39aa2dc145d43 (127.30.208.1:40045): tablet c01ce0292b5a4d6eb22e53faef90d47c (table test-table [id=7657fd003263427b8373b080452d6316]) successfully deleted
I20260807 19:42:03.963471 945 catalog_manager.cc:4841] TS ab40e6ac7585412cbf265a8bd66aa72d (127.30.208.2:41287): tablet c01ce0292b5a4d6eb22e53faef90d47c (table test-table [id=7657fd003263427b8373b080452d6316]) successfully deleted
Aug 07 19:42:03 dist-test-slave-vq2q krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131721, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:42:03.972645 1671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:03.944409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37948 (local address 127.30.208.1:40045)
0807 19:42:03.952037 (+ 7628us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:03.952044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:03.952072 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:42:03.952144 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:03.952150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:03.952251 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:03.952374 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:03.954331 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:03.955391 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:03.957418 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:03.958059 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:03.958190 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:03.966940 (+ 8750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:03.966957 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:03.966973 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:03.967015 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:03.969898 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:03.970559 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:03.970566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:03.970568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:03.970704 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:03.971089 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:03.971092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:03.971094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:03.971466 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:03.971756 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:03.972074 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 19:42:03.972327 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7472,"thread_start_us":7292,"threads_started":1}
W20260807 19:42:03.973356 1173 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37948
W20260807 19:42:03.974028 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 1034: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:42:03.974229 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 1034
I20260807 19:42:04.002254 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 1183
Aug 07 19:42:04 dist-test-slave-vq2q krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131721, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:42:04.019956 1672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:04.003007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52308 (local address 127.30.208.2:41287)
0807 19:42:04.003559 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:04.003566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:04.003586 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:42:04.003643 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:04.003648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:04.003722 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:04.003835 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:04.005083 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:04.006255 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:04.007640 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:04.008361 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:04.008489 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:04.013550 (+ 5061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:04.013569 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:04.013586 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:04.013628 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:04.016958 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:04.017915 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:04.017924 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:04.017927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:04.018106 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:04.018518 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:04.018521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:04.018523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:04.018886 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:04.019134 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:04.019482 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 19:42:04.019677 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":253,"threads_started":1}
W20260807 19:42:04.020664 1321 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52308
W20260807 19:42:04.021366 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 1183: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:42:04.021528 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 1183
I20260807 19:42:04.042235 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 1330
Aug 07 19:42:04 dist-test-slave-vq2q krb5kdc[914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131721, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:42:04.058635 1673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:04.043047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36418 (local address 127.30.208.3:34923)
0807 19:42:04.043619 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:04.043625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:04.043647 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:42:04.043705 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:04.043709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:04.043779 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:04.043889 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:04.045720 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:04.046860 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:04.048066 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:04.048496 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:04.048589 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:04.052904 (+ 4315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:04.052920 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:04.052937 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:04.052972 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:04.055724 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:04.056383 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:04.056390 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:04.056392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:04.056559 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:04.057015 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:04.057019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:04.057022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:04.057453 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:04.057754 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:04.058092 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 19:42:04.058311 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":273,"threads_started":1}
W20260807 19:42:04.059286 1471 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36418
W20260807 19:42:04.059904 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 1330: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:42:04.060056 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 1330
I20260807 19:42:04.083041 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 927
I20260807 19:42:04.084249 1002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:04.203089 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 927
2026-08-07T19:42:04Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9046 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:04 dist-test-slave-vq2q krb5kdc[1678](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:04 dist-test-slave-vq2q krb5kdc[1678](info): set up 2 sockets
Aug 07 19:42:04 dist-test-slave-vq2q krb5kdc[1678](info): commencing operation
krb5kdc: starting...
W20260807 19:42:05.272102 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:42:05 dist-test-slave-vq2q krb5kdc[1678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:42:05.477834 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:40941
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:40941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:05.818336 1691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:05.818630 1691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:05.818769 1691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:05.826193 1691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:05.826316 1691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:05.826367 1691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:05.826453 1691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:05.826502 1691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:05.836575 1691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:40941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:40941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:05.837960 1691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:05.839322 1691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:05.851572 1699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:05.851641 1696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:05.851619 1697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:05.852129 1691 server_base.cc:1034] running on GCE node
I20260807 19:42:05.852944 1691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:05.854415 1691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:05.855633 1691 hybrid_clock.cc:648] HybridClock initialized: now 1786131725855578 us; error 47 us; skew 500 ppm
I20260807 19:42:05.855970 1691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:05 dist-test-slave-vq2q krb5kdc[1678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131725, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:05.859117 1691 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:42:05.860602 1691 webserver.cc:460] Webserver started at http://127.30.208.62:40211/ using document root <none> and password file <none>
I20260807 19:42:05.861299 1691 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:05.861402 1691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:05.861663 1691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:05.864560 1691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "c85c3ecbf3c04dddb4ade3dedf9a1295"
format_stamp: "Formatted at 2026-08-07 19:42:05 on dist-test-slave-vq2q"
server_key: "f2d370025f5f8d96e35a0dc5b57b6933"
server_key_iv: "9d8b7d8fbb1dcdf246b6add338c064ee"
server_key_version: "encryptionkey@0"
I20260807 19:42:05.865455 1691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "c85c3ecbf3c04dddb4ade3dedf9a1295"
format_stamp: "Formatted at 2026-08-07 19:42:05 on dist-test-slave-vq2q"
server_key: "f2d370025f5f8d96e35a0dc5b57b6933"
server_key_iv: "9d8b7d8fbb1dcdf246b6add338c064ee"
server_key_version: "encryptionkey@0"
I20260807 19:42:05.871018 1691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:42:05.874439 1707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:05.875780 1691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:05.875942 1691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "c85c3ecbf3c04dddb4ade3dedf9a1295"
format_stamp: "Formatted at 2026-08-07 19:42:05 on dist-test-slave-vq2q"
server_key: "f2d370025f5f8d96e35a0dc5b57b6933"
server_key_iv: "9d8b7d8fbb1dcdf246b6add338c064ee"
server_key_version: "encryptionkey@0"
I20260807 19:42:05.876116 1691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:05.892591 1691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:05.896834 1691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:05.897089 1691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:05.915000 1691 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:40941
I20260807 19:42:05.915020 1770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:40941 every 8 connection(s)
I20260807 19:42:05.916411 1691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:05.919700 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 1691
I20260807 19:42:05.919895 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:42:05.920287 1771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:05.920559 31552 external_mini_cluster.cc:1428] Setting key d8f95a287575a7bcc97027ef9f514319
I20260807 19:42:05.934911 1771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295: Bootstrap starting.
Aug 07 19:42:05 dist-test-slave-vq2q krb5kdc[1678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:05.938207 1771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:05.939285 1771 log.cc:824] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:05.942209 1771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295: No bootstrap required, opened a new log
I20260807 19:42:05.943370 1774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:05.922458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40956 (local address 127.30.208.62:40941)
0807 19:42:05.923331 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:05.923343 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:05.923409 (+ 66us) server_negotiation.cc:407] Connection header received
0807 19:42:05.924981 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:05.925002 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:05.925461 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:05.925813 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:05.927620 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:05.928935 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:05.930737 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:05.931545 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:05.931688 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:05.936233 (+ 4545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:05.936241 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:05.936255 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:05.936297 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:05.939301 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:05.939971 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:05.939981 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:05.939986 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:05.940173 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:05.940557 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:05.940561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:05.940563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:05.941102 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:05.941437 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:05.941880 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 19:42:05.942323 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":378,"threads_started":1}
I20260807 19:42:05.950661 1771 raft_consensus.cc:348] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c85c3ecbf3c04dddb4ade3dedf9a1295" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40941 } }
I20260807 19:42:05.951122 1771 raft_consensus.cc:374] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:05.951233 1771 raft_consensus.cc:729] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c85c3ecbf3c04dddb4ade3dedf9a1295, State: Initialized, Role: FOLLOWER
I20260807 19:42:05.951794 1771 consensus_queue.cc:260] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c85c3ecbf3c04dddb4ade3dedf9a1295" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40941 } }
I20260807 19:42:05.952013 1771 raft_consensus.cc:388] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:05.952140 1771 raft_consensus.cc:482] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:05.952306 1771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:05.954536 1771 raft_consensus.cc:504] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c85c3ecbf3c04dddb4ade3dedf9a1295" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40941 } }
I20260807 19:42:05.955034 1771 leader_election.cc:302] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c85c3ecbf3c04dddb4ade3dedf9a1295; no voters:
I20260807 19:42:05.955605 1771 leader_election.cc:288] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:05.955734 1776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:05.956349 1776 raft_consensus.cc:686] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [term 1 LEADER]: Becoming Leader. State: Replica: c85c3ecbf3c04dddb4ade3dedf9a1295, State: Running, Role: LEADER
I20260807 19:42:05.956845 1776 consensus_queue.cc:237] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c85c3ecbf3c04dddb4ade3dedf9a1295" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40941 } }
I20260807 19:42:05.957768 1771 sys_catalog.cc:564] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:05.960048 1777 sys_catalog.cc:455] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c85c3ecbf3c04dddb4ade3dedf9a1295" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c85c3ecbf3c04dddb4ade3dedf9a1295" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40941 } } }
I20260807 19:42:05.960103 1778 sys_catalog.cc:455] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c85c3ecbf3c04dddb4ade3dedf9a1295. Latest consensus state: current_term: 1 leader_uuid: "c85c3ecbf3c04dddb4ade3dedf9a1295" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c85c3ecbf3c04dddb4ade3dedf9a1295" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40941 } } }
I20260807 19:42:05.960469 1777 sys_catalog.cc:458] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:05.960480 1778 sys_catalog.cc:458] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:05.962225 1784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:05.967443 1784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:05.976016 1784 catalog_manager.cc:1347] Generated new cluster ID: 9f2af38c742e4d2086104602b7c6f46a
I20260807 19:42:05.976138 1784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:05.988384 1784 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:05.990321 1784 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:05.999763 1784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c85c3ecbf3c04dddb4ade3dedf9a1295: Generated new TSK 0
I20260807 19:42:06.000952 1784 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:42:06.053058 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:40941
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:06.382812 1797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:06.383123 1797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:06.383260 1797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:06.390733 1797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:06.390867 1797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:06.391016 1797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:42:06.401556 1797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:40941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:06.402894 1797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:06.404196 1797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:06.416863 1803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:06.416944 1804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:06.416909 1807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:06.417788 1797 server_base.cc:1034] running on GCE node
I20260807 19:42:06.418424 1797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:06.419538 1797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:06.420776 1797 hybrid_clock.cc:648] HybridClock initialized: now 1786131726420723 us; error 52 us; skew 500 ppm
I20260807 19:42:06.421103 1797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:06 dist-test-slave-vq2q krb5kdc[1678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131726, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:06.424364 1797 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:42:06.425833 1797 webserver.cc:460] Webserver started at http://127.30.208.1:40415/ using document root <none> and password file <none>
I20260807 19:42:06.426509 1797 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:06.426615 1797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:06.426914 1797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:06.429831 1797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "cde0005b7a254fa89d3c7570af239a0b"
format_stamp: "Formatted at 2026-08-07 19:42:06 on dist-test-slave-vq2q"
server_key: "add9a1ae282a2a8b732d08d28784fde5"
server_key_iv: "00b9d329612bf195d5e42090a022d545"
server_key_version: "encryptionkey@0"
I20260807 19:42:06.430612 1797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "cde0005b7a254fa89d3c7570af239a0b"
format_stamp: "Formatted at 2026-08-07 19:42:06 on dist-test-slave-vq2q"
server_key: "add9a1ae282a2a8b732d08d28784fde5"
server_key_iv: "00b9d329612bf195d5e42090a022d545"
server_key_version: "encryptionkey@0"
I20260807 19:42:06.435747 1797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:42:06.439265 1816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:06.440615 1797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:42:06.440759 1797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "cde0005b7a254fa89d3c7570af239a0b"
format_stamp: "Formatted at 2026-08-07 19:42:06 on dist-test-slave-vq2q"
server_key: "add9a1ae282a2a8b732d08d28784fde5"
server_key_iv: "00b9d329612bf195d5e42090a022d545"
server_key_version: "encryptionkey@0"
I20260807 19:42:06.440950 1797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:06.468709 1797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:06.472963 1797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:06.473277 1797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:06.474248 1797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:06.475607 1797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:06.475679 1797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:06.475798 1797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:06.475852 1797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:06.507464 1797 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:43001
I20260807 19:42:06.507537 1936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:43001 every 8 connection(s)
I20260807 19:42:06.508744 1797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:06.515266 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 1797
I20260807 19:42:06.515465 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:06.516155 31552 external_mini_cluster.cc:1428] Setting key 87f38b84020000a1590722f8adaed7cf
Aug 07 19:42:06 dist-test-slave-vq2q krb5kdc[1678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131726, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:06.531564 1939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:06.511307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:47385 (local address 127.30.208.62:40941)
0807 19:42:06.511809 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:06.511816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:06.512850 (+ 1034us) server_negotiation.cc:407] Connection header received
0807 19:42:06.514800 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:06.514806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:06.514905 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:06.515042 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:06.516853 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:06.517855 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:06.520167 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:06.520604 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:06.520710 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:06.525039 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:06.525045 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:06.525049 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:06.525087 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:06.528239 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:06.529038 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:06.529045 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:06.529048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:06.529252 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:06.529670 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:06.529674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:06.529677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:06.530024 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:06.530158 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:06.531005 (+ 847us) server_negotiation.cc:299] Negotiation successful
0807 19:42:06.531227 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":229,"threads_started":1}
I20260807 19:42:06.532437 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:06.511886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:40941 (local address 127.30.208.1:47385)
0807 19:42:06.512662 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:06.512718 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:06.514475 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:06.515289 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:06.515297 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:06.516003 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:06.516634 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:06.516654 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:06.518064 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:06.518068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:06.519956 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:06.519975 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:06.520855 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:06.520862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:06.521142 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:06.521903 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:06.521934 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:06.524732 (+ 2798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:06.528412 (+ 3680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:06.528420 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:06.528435 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:06.528903 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:06.529362 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:06.529365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:06.529367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:06.529564 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:06.530203 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:06.530211 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:06.530560 (+ 349us) client_negotiation.cc:769] Sending connection context
0807 19:42:06.530852 (+ 292us) client_negotiation.cc:240] Negotiation successful
0807 19:42:06.531279 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":322,"threads_started":1}
I20260807 19:42:06.534001 1937 heartbeater.cc:347] Connected to a master server at 127.30.208.62:40941
I20260807 19:42:06.534356 1937 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:06.535250 1937 heartbeater.cc:511] Master 127.30.208.62:40941 requested a full tablet report, sending...
I20260807 19:42:06.537571 1727 ts_manager.cc:194] Registered new tserver with Master: cde0005b7a254fa89d3c7570af239a0b (127.30.208.1:43001)
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
I20260807 19:42:06.539078 1727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:47385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:42:06.566125 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:40941
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:06.889925 1943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:06.890199 1943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:06.890295 1943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:06.897360 1943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:06.897467 1943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:06.897581 1943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:42:06.907322 1943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:40941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:06.908581 1943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:06.909893 1943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:06.923676 1949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:06.923658 1950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:06.924430 1952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:06.925340 1943 server_base.cc:1034] running on GCE node
I20260807 19:42:06.926059 1943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:06.927104 1943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:06.928314 1943 hybrid_clock.cc:648] HybridClock initialized: now 1786131726928255 us; error 54 us; skew 500 ppm
I20260807 19:42:06.928627 1943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:06 dist-test-slave-vq2q krb5kdc[1678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131726, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:06.932034 1943 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:42:06.934075 1943 webserver.cc:460] Webserver started at http://127.30.208.2:43439/ using document root <none> and password file <none>
I20260807 19:42:06.934804 1943 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:06.934931 1943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:06.935230 1943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:06.938275 1943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "1927cfd860e64f738c454a4d5c27ceb6"
format_stamp: "Formatted at 2026-08-07 19:42:06 on dist-test-slave-vq2q"
server_key: "823c2ceb6c7669cffee431beb3759fbf"
server_key_iv: "ffbf63c5a4cead7941bfbd6e2a316237"
server_key_version: "encryptionkey@0"
I20260807 19:42:06.939109 1943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "1927cfd860e64f738c454a4d5c27ceb6"
format_stamp: "Formatted at 2026-08-07 19:42:06 on dist-test-slave-vq2q"
server_key: "823c2ceb6c7669cffee431beb3759fbf"
server_key_iv: "ffbf63c5a4cead7941bfbd6e2a316237"
server_key_version: "encryptionkey@0"
I20260807 19:42:06.944545 1943 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 19:42:06.948302 1960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:06.950109 1943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:06.950294 1943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "1927cfd860e64f738c454a4d5c27ceb6"
format_stamp: "Formatted at 2026-08-07 19:42:06 on dist-test-slave-vq2q"
server_key: "823c2ceb6c7669cffee431beb3759fbf"
server_key_iv: "ffbf63c5a4cead7941bfbd6e2a316237"
server_key_version: "encryptionkey@0"
I20260807 19:42:06.950507 1943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:06.963320 1943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:06.967622 1943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:06.967916 1943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:06.968940 1943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:06.970367 1943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:06.970448 1943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:06.970571 1943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:06.970625 1943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:07.001147 1943 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:45057
I20260807 19:42:07.001223 2078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:45057 every 8 connection(s)
I20260807 19:42:07.002456 1943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:07.006320 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 1943
I20260807 19:42:07.006496 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:07.006914 31552 external_mini_cluster.cc:1428] Setting key a81606c1465c43e5d4ce1b94995fb595
Aug 07 19:42:07 dist-test-slave-vq2q krb5kdc[1678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131726, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:07.022488 1939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:07.004914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:38861 (local address 127.30.208.62:40941)
0807 19:42:07.005226 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:07.005232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:07.006294 (+ 1062us) server_negotiation.cc:407] Connection header received
0807 19:42:07.008195 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:07.008200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:07.008277 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:07.008365 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:07.009849 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:07.010610 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:07.012178 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:07.012767 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:07.012901 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:07.016494 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:07.016505 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:07.016509 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:07.016549 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:07.019182 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:07.020046 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:07.020052 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:07.020055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:07.020214 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:07.020691 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:07.020694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:07.020696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:07.021054 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:07.021167 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:07.022054 (+ 887us) server_negotiation.cc:299] Negotiation successful
0807 19:42:07.022242 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260807 19:42:07.023828 2081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:07.005417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:40941 (local address 127.30.208.2:38861)
0807 19:42:07.006132 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:07.006177 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:07.007936 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:07.008600 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:07.008607 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:07.009113 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:07.009694 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:07.009710 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:07.010753 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:07.010757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:07.012005 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:07.012028 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:07.013010 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:07.013016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:07.013257 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:07.013879 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:07.013902 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:07.016250 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:07.019330 (+ 3080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:07.019339 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:07.019355 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:07.019918 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:07.020317 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:07.020321 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:07.020324 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:07.020572 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:07.021203 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:07.021211 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:07.021630 (+ 419us) client_negotiation.cc:769] Sending connection context
0807 19:42:07.021939 (+ 309us) client_negotiation.cc:240] Negotiation successful
0807 19:42:07.022516 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":272,"threads_started":1}
I20260807 19:42:07.025381 2079 heartbeater.cc:347] Connected to a master server at 127.30.208.62:40941
I20260807 19:42:07.025774 2079 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:07.026688 2079 heartbeater.cc:511] Master 127.30.208.62:40941 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
I20260807 19:42:07.028339 1727 ts_manager.cc:194] Registered new tserver with Master: 1927cfd860e64f738c454a4d5c27ceb6 (127.30.208.2:45057)
I20260807 19:42:07.029158 1727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:38861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:42:07.054802 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:40941
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:32999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:07.380455 2085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:07.380757 2085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:07.380890 2085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:07.388178 2085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:07.388316 2085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:07.388486 2085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:42:07.398528 2085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:32999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:40941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:07.399857 2085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:07.401166 2085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:07.414026 2093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:07.414026 2090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:07.413983 2091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:07.414790 2085 server_base.cc:1034] running on GCE node
I20260807 19:42:07.415401 2085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:07.416342 2085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:07.417557 2085 hybrid_clock.cc:648] HybridClock initialized: now 1786131727417503 us; error 43 us; skew 500 ppm
I20260807 19:42:07.417877 2085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:07 dist-test-slave-vq2q krb5kdc[1678](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131727, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:07.420907 2085 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:42:07.422446 2085 webserver.cc:460] Webserver started at http://127.30.208.3:32901/ using document root <none> and password file <none>
I20260807 19:42:07.423148 2085 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:07.423259 2085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:07.423631 2085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:07.426622 2085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "53a2a78817b44690a7412f8946c27e5b"
format_stamp: "Formatted at 2026-08-07 19:42:07 on dist-test-slave-vq2q"
server_key: "b2dc3540162eaa0aab1bfad80388668e"
server_key_iv: "cc5ce8e74ba1d8f61f4ea65e4c864319"
server_key_version: "encryptionkey@0"
I20260807 19:42:07.427397 2085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "53a2a78817b44690a7412f8946c27e5b"
format_stamp: "Formatted at 2026-08-07 19:42:07 on dist-test-slave-vq2q"
server_key: "b2dc3540162eaa0aab1bfad80388668e"
server_key_iv: "cc5ce8e74ba1d8f61f4ea65e4c864319"
server_key_version: "encryptionkey@0"
I20260807 19:42:07.432595 2085 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 19:42:07.436007 2100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:07.437347 2085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:07.437490 2085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "53a2a78817b44690a7412f8946c27e5b"
format_stamp: "Formatted at 2026-08-07 19:42:07 on dist-test-slave-vq2q"
server_key: "b2dc3540162eaa0aab1bfad80388668e"
server_key_iv: "cc5ce8e74ba1d8f61f4ea65e4c864319"
server_key_version: "encryptionkey@0"
I20260807 19:42:07.437671 2085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:07.452770 2085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:07.456849 2085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:07.457091 2085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:07.458076 2085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:07.459473 2085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:07.459545 2085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:07.459636 2085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:07.459684 2085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:07.488878 2085 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:34399
I20260807 19:42:07.488932 2218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:34399 every 8 connection(s)
I20260807 19:42:07.490161 2085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:07.494614 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 2085
I20260807 19:42:07.494755 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:07.495172 31552 external_mini_cluster.cc:1428] Setting key 98f61f6a3c0480208131d0f229a24ca4
Aug 07 19:42:07 dist-test-slave-vq2q krb5kdc[1678](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131727, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:07.510771 1939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:07.492729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:56271 (local address 127.30.208.62:40941)
0807 19:42:07.492913 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:07.492917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:07.494145 (+ 1228us) server_negotiation.cc:407] Connection header received
0807 19:42:07.496269 (+ 2124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:07.496274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:07.496393 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:07.496495 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:07.498305 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:07.499058 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:07.500573 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:07.500989 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:07.501081 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:07.504390 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:07.504395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:07.504398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:07.504433 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:07.506917 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:07.507906 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:07.507923 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:07.507925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:07.508078 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:07.508682 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:07.508687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:07.508691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:07.509027 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:07.509199 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:07.510197 (+ 998us) server_negotiation.cc:299] Negotiation successful
0807 19:42:07.510450 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:42:07.512104 2221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:07.493213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:40941 (local address 127.30.208.3:56271)
0807 19:42:07.493980 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:07.494020 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:07.495909 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:07.496859 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:07.496871 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:07.497472 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:07.498143 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:07.498173 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:07.499218 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:07.499222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:07.500441 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:07.500454 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:07.501159 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:07.501206 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:07.501411 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:07.502077 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:07.502099 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:07.504200 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:07.507095 (+ 2895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:07.507104 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:07.507122 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:07.507725 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:07.508225 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:07.508230 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:07.508234 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:07.508509 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:07.509347 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:07.509360 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:07.509771 (+ 411us) client_negotiation.cc:769] Sending connection context
0807 19:42:07.510074 (+ 303us) client_negotiation.cc:240] Negotiation successful
0807 19:42:07.510765 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":312,"threads_started":1}
I20260807 19:42:07.513468 2219 heartbeater.cc:347] Connected to a master server at 127.30.208.62:40941
I20260807 19:42:07.513777 2219 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:07.514505 2219 heartbeater.cc:511] Master 127.30.208.62:40941 requested a full tablet report, sending...
I20260807 19:42:07.515903 1727 ts_manager.cc:194] Registered new tserver with Master: 53a2a78817b44690a7412f8946c27e5b (127.30.208.3:34399)
I20260807 19:42:07.516685 1727 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:56271
I20260807 19:42:07.522804 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:42:07.542408 1937 heartbeater.cc:503] Master 127.30.208.62:40941 was elected leader, sending a full tablet report...
W20260807 19:42:07.558506 1939 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:07.555931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40960 (local address 127.30.208.62:40941)
0807 19:42:07.556108 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:07.556114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:07.556553 (+ 439us) server_negotiation.cc:407] Connection header received
0807 19:42:07.556738 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:07.556742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:07.556813 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:07.556906 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:07.558279 (+ 1373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40960: BlockingRecv error: recv got EOF from 127.0.0.1:40960 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 19:42:07.561672 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 1797
W20260807 19:42:07.564338 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 1797: Not authorized: Client connection negotiation failed: client connection to 127.30.208.1:43001: server requires authentication, but client does not have Kerberos credentials available
I20260807 19:42:07.564574 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 1797
I20260807 19:42:07.580936 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 1943
W20260807 19:42:07.583618 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 1943: Not authorized: Client connection negotiation failed: client connection to 127.30.208.2:45057: server requires authentication, but client does not have Kerberos credentials available
I20260807 19:42:07.583772 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 1943
I20260807 19:42:07.599582 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 2085
W20260807 19:42:07.602372 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 2085: Not authorized: Client connection negotiation failed: client connection to 127.30.208.3:34399: server requires authentication, but client does not have Kerberos credentials available
I20260807 19:42:07.602521 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 2085
I20260807 19:42:07.619059 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 1691
I20260807 19:42:07.620281 1761 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:07.725107 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 1691
2026-08-07T19:42:07Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3519 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:07 dist-test-slave-vq2q krb5kdc[2237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:07 dist-test-slave-vq2q krb5kdc[2237](info): set up 2 sockets
Aug 07 19:42:07 dist-test-slave-vq2q krb5kdc[2237](info): commencing operation
krb5kdc: starting...
W20260807 19:42:08.783689 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:42:08 dist-test-slave-vq2q krb5kdc[2237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:42:08.997632 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:40523
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:40523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:09.325525 2251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:09.325901 2251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:09.326053 2251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:09.333348 2251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:09.333456 2251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:09.333526 2251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:09.333607 2251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:09.333675 2251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:09.343786 2251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:40523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:40523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:09.345203 2251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:09.346567 2251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:09.358296 2257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:09.358384 2260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:09.358539 2258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:09.359794 2251 server_base.cc:1034] running on GCE node
I20260807 19:42:09.360626 2251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:09.362223 2251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:09.363435 2251 hybrid_clock.cc:648] HybridClock initialized: now 1786131729363397 us; error 28 us; skew 500 ppm
I20260807 19:42:09.363782 2251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:09 dist-test-slave-vq2q krb5kdc[2237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131729, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:09.373224 2251 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:42:09.374886 2251 webserver.cc:460] Webserver started at http://127.30.208.62:34989/ using document root <none> and password file <none>
I20260807 19:42:09.375588 2251 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:09.375718 2251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:09.376035 2251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:09.379204 2251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "e23cd6247b424be7b56abbf8e4a8081b"
format_stamp: "Formatted at 2026-08-07 19:42:09 on dist-test-slave-vq2q"
server_key: "7c31fe27ec15af1863fffa5fa24a8ee7"
server_key_iv: "e8c861a950869f2ba49d994f194dd4a5"
server_key_version: "encryptionkey@0"
I20260807 19:42:09.380076 2251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "e23cd6247b424be7b56abbf8e4a8081b"
format_stamp: "Formatted at 2026-08-07 19:42:09 on dist-test-slave-vq2q"
server_key: "7c31fe27ec15af1863fffa5fa24a8ee7"
server_key_iv: "e8c861a950869f2ba49d994f194dd4a5"
server_key_version: "encryptionkey@0"
I20260807 19:42:09.386018 2251 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 19:42:09.389843 2269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:09.391283 2251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:42:09.391464 2251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "e23cd6247b424be7b56abbf8e4a8081b"
format_stamp: "Formatted at 2026-08-07 19:42:09 on dist-test-slave-vq2q"
server_key: "7c31fe27ec15af1863fffa5fa24a8ee7"
server_key_iv: "e8c861a950869f2ba49d994f194dd4a5"
server_key_version: "encryptionkey@0"
I20260807 19:42:09.391669 2251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:09.416069 2251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:09.420531 2251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:09.420826 2251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:09.438920 2251 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:40523
I20260807 19:42:09.438993 2330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:40523 every 8 connection(s)
I20260807 19:42:09.440299 2251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:09.443943 2331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:09.449496 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 2251
I20260807 19:42:09.449719 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:42:09.450316 31552 external_mini_cluster.cc:1428] Setting key 561bd40dc63f853249d5d0758860a4cd
I20260807 19:42:09.455423 2331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b: Bootstrap starting.
I20260807 19:42:09.459167 2331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:09.460474 2331 log.cc:824] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:09.463464 2331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b: No bootstrap required, opened a new log
Aug 07 19:42:09 dist-test-slave-vq2q krb5kdc[2237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:09.473101 2331 raft_consensus.cc:348] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e23cd6247b424be7b56abbf8e4a8081b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40523 } }
I20260807 19:42:09.473657 2331 raft_consensus.cc:374] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:09.473767 2331 raft_consensus.cc:729] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23cd6247b424be7b56abbf8e4a8081b, State: Initialized, Role: FOLLOWER
I20260807 19:42:09.474490 2331 consensus_queue.cc:260] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e23cd6247b424be7b56abbf8e4a8081b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40523 } }
I20260807 19:42:09.474741 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:09.452131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51544 (local address 127.30.208.62:40523)
0807 19:42:09.452959 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:09.452974 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:09.453031 (+ 57us) server_negotiation.cc:407] Connection header received
0807 19:42:09.455419 (+ 2388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:09.455447 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:09.455958 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:09.456391 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:09.457893 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:09.459839 (+ 1946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:09.461313 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:09.462171 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:09.462349 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:09.466070 (+ 3721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:09.466081 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:09.466104 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:09.466146 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:09.470082 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:09.470783 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:09.470792 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:09.470801 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:09.471020 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:09.471361 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:09.471366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:09.471368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:09.472198 (+ 830us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:09.472477 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:09.472858 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 19:42:09.473324 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":303,"threads_started":1}
I20260807 19:42:09.474789 2331 raft_consensus.cc:388] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:09.474916 2331 raft_consensus.cc:482] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:09.475145 2331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:09.477671 2331 raft_consensus.cc:504] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e23cd6247b424be7b56abbf8e4a8081b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40523 } }
I20260807 19:42:09.478199 2331 leader_election.cc:302] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e23cd6247b424be7b56abbf8e4a8081b; no voters:
I20260807 19:42:09.478576 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:09.478840 2331 leader_election.cc:288] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:09.478964 2336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:09.480727 2331 sys_catalog.cc:564] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:09.485457 2336 raft_consensus.cc:686] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [term 1 LEADER]: Becoming Leader. State: Replica: e23cd6247b424be7b56abbf8e4a8081b, State: Running, Role: LEADER
I20260807 19:42:09.486061 2336 consensus_queue.cc:237] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e23cd6247b424be7b56abbf8e4a8081b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40523 } }
I20260807 19:42:09.489331 2337 sys_catalog.cc:455] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e23cd6247b424be7b56abbf8e4a8081b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e23cd6247b424be7b56abbf8e4a8081b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40523 } } }
I20260807 19:42:09.489749 2337 sys_catalog.cc:458] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:09.489355 2338 sys_catalog.cc:455] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e23cd6247b424be7b56abbf8e4a8081b. Latest consensus state: current_term: 1 leader_uuid: "e23cd6247b424be7b56abbf8e4a8081b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e23cd6247b424be7b56abbf8e4a8081b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40523 } } }
I20260807 19:42:09.490489 2338 sys_catalog.cc:458] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b [sys.catalog]: This master's current role is: LEADER
W20260807 19:42:09.502573 2351 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:42:09.502717 2351 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:42:09.502907 2352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:09.505067 2352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:09.514478 2352 catalog_manager.cc:1347] Generated new cluster ID: 8ee51010951143eda65cb0c24a142dfc
I20260807 19:42:09.514647 2352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:09.532940 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:09.523378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51550 (local address 127.30.208.62:40523)
0807 19:42:09.523601 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:09.523608 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:09.524244 (+ 636us) server_negotiation.cc:407] Connection header received
0807 19:42:09.524450 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:09.524455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:09.524540 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:09.524646 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:09.526011 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:09.527073 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:09.528990 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:09.529702 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:09.529831 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:09.531087 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:09.531093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:09.531098 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:09.531930 (+ 832us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:09.532009 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:09.532048 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:09.532146 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:09.532303 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 19:42:09.532403 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 19:42:09.544332 2352 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:09.545811 2352 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:09.553033 2352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e23cd6247b424be7b56abbf8e4a8081b: Generated new TSK 0
I20260807 19:42:09.553879 2352 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:09.563010 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 2251
I20260807 19:42:09.564245 2325 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:09.659907 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 2251
2026-08-07T19:42:09Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1930 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:09 dist-test-slave-vq2q krb5kdc[2366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:09 dist-test-slave-vq2q krb5kdc[2366](info): set up 2 sockets
Aug 07 19:42:09 dist-test-slave-vq2q krb5kdc[2366](info): commencing operation
krb5kdc: starting...
W20260807 19:42:10.748507 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:42:10 dist-test-slave-vq2q krb5kdc[2366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:42:10.969694 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:34155
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:34155
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:11.295485 2379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:11.295912 2379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:11.296046 2379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:11.303406 2379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:11.303530 2379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:11.303581 2379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:11.303661 2379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:11.303720 2379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:11.313694 2379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:34155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:34155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:11.315022 2379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:11.316341 2379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:11.327757 2388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:11.327807 2386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:11.327998 2385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:11.329023 2379 server_base.cc:1034] running on GCE node
I20260807 19:42:11.329857 2379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:11.331298 2379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:11.332525 2379 hybrid_clock.cc:648] HybridClock initialized: now 1786131731332479 us; error 50 us; skew 500 ppm
I20260807 19:42:11.332902 2379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:11 dist-test-slave-vq2q krb5kdc[2366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131731, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:11.335971 2379 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:42:11.337471 2379 webserver.cc:460] Webserver started at http://127.30.208.62:40653/ using document root <none> and password file <none>
I20260807 19:42:11.338125 2379 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:11.338243 2379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:11.338528 2379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:11.341507 2379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "d673f06e799a451e9daa42d809b23abf"
format_stamp: "Formatted at 2026-08-07 19:42:11 on dist-test-slave-vq2q"
server_key: "0c7e6c6f09dbc4cdb0f0718f7ad88634"
server_key_iv: "472f039653072cce70d33919b1ae64c0"
server_key_version: "encryptionkey@0"
I20260807 19:42:11.342267 2379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "d673f06e799a451e9daa42d809b23abf"
format_stamp: "Formatted at 2026-08-07 19:42:11 on dist-test-slave-vq2q"
server_key: "0c7e6c6f09dbc4cdb0f0718f7ad88634"
server_key_iv: "472f039653072cce70d33919b1ae64c0"
server_key_version: "encryptionkey@0"
I20260807 19:42:11.347358 2379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:42:11.350873 2395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:11.352208 2379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:42:11.352373 2379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "d673f06e799a451e9daa42d809b23abf"
format_stamp: "Formatted at 2026-08-07 19:42:11 on dist-test-slave-vq2q"
server_key: "0c7e6c6f09dbc4cdb0f0718f7ad88634"
server_key_iv: "472f039653072cce70d33919b1ae64c0"
server_key_version: "encryptionkey@0"
I20260807 19:42:11.352557 2379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:11.370052 2379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:11.373965 2379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:11.374240 2379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:11.392375 2379 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:34155
I20260807 19:42:11.392396 2452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:34155 every 8 connection(s)
I20260807 19:42:11.393796 2379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:11.397598 2453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:11.400915 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 2379
I20260807 19:42:11.401153 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:42:11.401962 31552 external_mini_cluster.cc:1428] Setting key 2654464523f1eee79ada5ba550f2ac1e
I20260807 19:42:11.410646 2453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf: Bootstrap starting.
I20260807 19:42:11.414525 2453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:11.415818 2453 log.cc:824] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:42:11 dist-test-slave-vq2q krb5kdc[2366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:11.418931 2453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf: No bootstrap required, opened a new log
I20260807 19:42:11.424589 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:11.404039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54102 (local address 127.30.208.62:34155)
0807 19:42:11.404799 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:11.404810 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:11.404851 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:42:11.406518 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:11.406540 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:11.406942 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:11.407265 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:11.408901 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:11.410185 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:11.411865 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:11.412626 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:11.412772 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:11.417566 (+ 4794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:11.417574 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:11.417589 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:11.417624 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:11.420543 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:11.421253 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:11.421259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:11.421269 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:11.421456 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:11.421830 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:11.421834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:11.421836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:11.422387 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:11.422707 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:11.423141 (+ 434us) server_negotiation.cc:299] Negotiation successful
0807 19:42:11.423561 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":353,"threads_started":1}
I20260807 19:42:11.427001 2453 raft_consensus.cc:348] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d673f06e799a451e9daa42d809b23abf" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34155 } }
I20260807 19:42:11.427465 2453 raft_consensus.cc:374] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:11.427572 2453 raft_consensus.cc:729] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d673f06e799a451e9daa42d809b23abf, State: Initialized, Role: FOLLOWER
I20260807 19:42:11.428103 2453 consensus_queue.cc:260] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d673f06e799a451e9daa42d809b23abf" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34155 } }
I20260807 19:42:11.428328 2453 raft_consensus.cc:388] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:11.428455 2453 raft_consensus.cc:482] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:11.428597 2453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:11.430900 2453 raft_consensus.cc:504] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d673f06e799a451e9daa42d809b23abf" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34155 } }
I20260807 19:42:11.431368 2453 leader_election.cc:302] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d673f06e799a451e9daa42d809b23abf; no voters:
I20260807 19:42:11.431882 2453 leader_election.cc:288] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:11.432047 2459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:11.433333 2459 raft_consensus.cc:686] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [term 1 LEADER]: Becoming Leader. State: Replica: d673f06e799a451e9daa42d809b23abf, State: Running, Role: LEADER
I20260807 19:42:11.433820 2459 consensus_queue.cc:237] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d673f06e799a451e9daa42d809b23abf" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34155 } }
I20260807 19:42:11.434017 2453 sys_catalog.cc:564] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:11.436461 2462 sys_catalog.cc:455] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [sys.catalog]: SysCatalogTable state changed. Reason: New leader d673f06e799a451e9daa42d809b23abf. Latest consensus state: current_term: 1 leader_uuid: "d673f06e799a451e9daa42d809b23abf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d673f06e799a451e9daa42d809b23abf" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34155 } } }
I20260807 19:42:11.436573 2460 sys_catalog.cc:455] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d673f06e799a451e9daa42d809b23abf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d673f06e799a451e9daa42d809b23abf" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34155 } } }
I20260807 19:42:11.436892 2462 sys_catalog.cc:458] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:11.436928 2460 sys_catalog.cc:458] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:11.439699 2468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:11.444854 2468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:11.453907 2468 catalog_manager.cc:1347] Generated new cluster ID: cc9249b0fcb64b9da1ba8dcd3df1d028
I20260807 19:42:11.454034 2468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:11.470072 2468 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:11.471434 2468 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:11.482398 2468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d673f06e799a451e9daa42d809b23abf: Generated new TSK 0
I20260807 19:42:11.483224 2468 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:42:11.542373 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34155
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:11.895076 2483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:11.895356 2483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:11.895481 2483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:11.902891 2483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:11.903021 2483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:11.903168 2483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:42:11.913116 2483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:11.914495 2483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:11.915830 2483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:11.929003 2492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:11.929075 2491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:11.929026 2494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:11.929965 2483 server_base.cc:1034] running on GCE node
I20260807 19:42:11.930583 2483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:11.931682 2483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:11.932911 2483 hybrid_clock.cc:648] HybridClock initialized: now 1786131731932862 us; error 40 us; skew 500 ppm
I20260807 19:42:11.933291 2483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:11 dist-test-slave-vq2q krb5kdc[2366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131731, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:11.936793 2483 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:42:11.938344 2483 webserver.cc:460] Webserver started at http://127.30.208.1:38321/ using document root <none> and password file <none>
I20260807 19:42:11.939039 2483 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:11.939177 2483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:11.939549 2483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:11.942662 2483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "fe9e3dd76c7144ab9991e263370b4c4b"
format_stamp: "Formatted at 2026-08-07 19:42:11 on dist-test-slave-vq2q"
server_key: "3411318d9a232927818ceaca55ade128"
server_key_iv: "d4125cc525fcb051ebceaff8e1555806"
server_key_version: "encryptionkey@0"
I20260807 19:42:11.943461 2483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "fe9e3dd76c7144ab9991e263370b4c4b"
format_stamp: "Formatted at 2026-08-07 19:42:11 on dist-test-slave-vq2q"
server_key: "3411318d9a232927818ceaca55ade128"
server_key_iv: "d4125cc525fcb051ebceaff8e1555806"
server_key_version: "encryptionkey@0"
I20260807 19:42:11.949286 2483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:42:11.952893 2503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:11.954274 2483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:11.954450 2483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "fe9e3dd76c7144ab9991e263370b4c4b"
format_stamp: "Formatted at 2026-08-07 19:42:11 on dist-test-slave-vq2q"
server_key: "3411318d9a232927818ceaca55ade128"
server_key_iv: "d4125cc525fcb051ebceaff8e1555806"
server_key_version: "encryptionkey@0"
I20260807 19:42:11.954653 2483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:11.987283 2483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:11.991523 2483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:11.991822 2483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:11.992796 2483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:11.994202 2483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:11.994283 2483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:11.994413 2483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:11.994467 2483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:12.025909 2483 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:38949
I20260807 19:42:12.025956 2628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:38949 every 8 connection(s)
I20260807 19:42:12.027259 2483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:12.037161 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 2483
I20260807 19:42:12.037447 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:12.038146 31552 external_mini_cluster.cc:1428] Setting key 1e3b1ba7b009030daba6c0e07f87cb02
Aug 07 19:42:12 dist-test-slave-vq2q krb5kdc[2366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131731, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:12.048519 2632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:12.030019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:33093 (local address 127.30.208.62:34155)
0807 19:42:12.030517 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:12.030521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:12.031562 (+ 1041us) server_negotiation.cc:407] Connection header received
0807 19:42:12.033485 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:12.033491 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:12.033575 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:12.033674 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:12.035287 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:12.036031 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:12.037747 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:12.038167 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:12.038286 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:12.042635 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:12.042641 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:12.042644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:12.042691 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:12.045396 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:12.046166 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:12.046170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:12.046173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:12.046302 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:12.046745 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:12.046748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:12.046750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:12.047040 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:12.047185 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:12.047911 (+ 726us) server_negotiation.cc:299] Negotiation successful
0807 19:42:12.048143 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":241,"threads_started":1}
I20260807 19:42:12.049546 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:12.030604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34155 (local address 127.30.208.1:33093)
0807 19:42:12.031393 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:12.031446 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:12.033145 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:12.033961 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:12.033970 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:12.034546 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:12.035111 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:12.035128 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:12.036197 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:12.036201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:12.037600 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:12.037615 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:12.038420 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:12.038428 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:12.038665 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:12.039869 (+ 1204us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:12.039907 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:12.042389 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:12.045544 (+ 3155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:12.045554 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:12.045571 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:12.046031 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:12.046416 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:12.046421 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:12.046424 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:12.046624 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:12.047168 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:12.047176 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:12.047534 (+ 358us) client_negotiation.cc:769] Sending connection context
0807 19:42:12.047778 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 19:42:12.048319 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":282,"threads_started":1}
I20260807 19:42:12.051121 2629 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34155
I20260807 19:42:12.051409 2629 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:12.052136 2629 heartbeater.cc:511] Master 127.30.208.62:34155 requested a full tablet report, sending...
I20260807 19:42:12.054451 2414 ts_manager.cc:194] Registered new tserver with Master: fe9e3dd76c7144ab9991e263370b4c4b (127.30.208.1:38949)
I20260807 19:42:12.056532 2414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:33093
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:42:12.086486 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34155
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:12.414652 2636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:12.414939 2636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:12.415076 2636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:12.422386 2636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:12.422528 2636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:12.422680 2636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:42:12.432818 2636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:12.434266 2636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:12.435638 2636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:12.448678 2642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:12.448712 2646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:12.448784 2644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:12.449379 2636 server_base.cc:1034] running on GCE node
I20260807 19:42:12.449951 2636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:12.450827 2636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:12.452060 2636 hybrid_clock.cc:648] HybridClock initialized: now 1786131732451999 us; error 48 us; skew 500 ppm
I20260807 19:42:12.452419 2636 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:12 dist-test-slave-vq2q krb5kdc[2366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131732, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:12.455626 2636 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:42:12.457141 2636 webserver.cc:460] Webserver started at http://127.30.208.2:34357/ using document root <none> and password file <none>
I20260807 19:42:12.457828 2636 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:12.457934 2636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:12.458251 2636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:12.461407 2636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "14a05bdcbf0b4fdeb0f1b8ff4676de12"
format_stamp: "Formatted at 2026-08-07 19:42:12 on dist-test-slave-vq2q"
server_key: "099ff5ab8dc9960b8e7f4d542871327e"
server_key_iv: "10fc95f29f6bc765ab8c9d25f9fc1e33"
server_key_version: "encryptionkey@0"
I20260807 19:42:12.462212 2636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "14a05bdcbf0b4fdeb0f1b8ff4676de12"
format_stamp: "Formatted at 2026-08-07 19:42:12 on dist-test-slave-vq2q"
server_key: "099ff5ab8dc9960b8e7f4d542871327e"
server_key_iv: "10fc95f29f6bc765ab8c9d25f9fc1e33"
server_key_version: "encryptionkey@0"
I20260807 19:42:12.467464 2636 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:42:12.470909 2655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:12.472189 2636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:42:12.472340 2636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "14a05bdcbf0b4fdeb0f1b8ff4676de12"
format_stamp: "Formatted at 2026-08-07 19:42:12 on dist-test-slave-vq2q"
server_key: "099ff5ab8dc9960b8e7f4d542871327e"
server_key_iv: "10fc95f29f6bc765ab8c9d25f9fc1e33"
server_key_version: "encryptionkey@0"
I20260807 19:42:12.472539 2636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:12.504771 2636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:12.509107 2636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:12.509434 2636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:12.510447 2636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:12.511801 2636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:12.511873 2636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:12.511981 2636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:12.512036 2636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:12.542958 2636 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:37651
I20260807 19:42:12.543015 2776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:37651 every 8 connection(s)
I20260807 19:42:12.544230 2636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:12.547828 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 2636
I20260807 19:42:12.548038 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:12.548620 31552 external_mini_cluster.cc:1428] Setting key 23b5df81a7e3bc21a455677e025b1854
Aug 07 19:42:12 dist-test-slave-vq2q krb5kdc[2366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131732, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:12.564563 2632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:12.546776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:43623 (local address 127.30.208.62:34155)
0807 19:42:12.547004 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:12.547010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:12.548362 (+ 1352us) server_negotiation.cc:407] Connection header received
0807 19:42:12.550517 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:12.550522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:12.550604 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:12.550686 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:12.552200 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:12.552907 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:12.554479 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:12.554889 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:12.555000 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:12.558822 (+ 3822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:12.558827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:12.558831 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:12.558865 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:12.561456 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:12.562211 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:12.562215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:12.562218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:12.562346 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:12.562793 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:12.562796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:12.562798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:12.563074 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:12.563206 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:12.563924 (+ 718us) server_negotiation.cc:299] Negotiation successful
0807 19:42:12.564174 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:42:12.565603 2779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:12.547281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34155 (local address 127.30.208.2:43623)
0807 19:42:12.548180 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:12.548231 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:12.550207 (+ 1976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:12.550951 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:12.550959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:12.551463 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:12.552036 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:12.552053 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:12.553058 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:12.553063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:12.554339 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:12.554351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:12.555107 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:12.555115 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:12.555348 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:12.556035 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:12.556058 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:12.558583 (+ 2525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:12.561604 (+ 3021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:12.561614 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:12.561632 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:12.562081 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:12.562463 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:12.562468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:12.562472 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:12.562676 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:12.563197 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:12.563204 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:12.563558 (+ 354us) client_negotiation.cc:769] Sending connection context
0807 19:42:12.563801 (+ 243us) client_negotiation.cc:240] Negotiation successful
0807 19:42:12.564398 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":411,"threads_started":1}
I20260807 19:42:12.567103 2777 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34155
I20260807 19:42:12.567476 2777 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
I20260807 19:42:12.568516 2777 heartbeater.cc:511] Master 127.30.208.62:34155 requested a full tablet report, sending...
I20260807 19:42:12.570268 2414 ts_manager.cc:194] Registered new tserver with Master: 14a05bdcbf0b4fdeb0f1b8ff4676de12 (127.30.208.2:37651)
I20260807 19:42:12.571066 2414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:43623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:42:12.596927 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34155
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:12.936187 2782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:12.936549 2782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:12.936668 2782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:12.944274 2782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:12.944402 2782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:12.944530 2782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:42:12.954734 2782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34155
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:12.956086 2782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:12.957490 2782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:12.971710 2789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:12.971820 2788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:12.971742 2791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:12.972431 2782 server_base.cc:1034] running on GCE node
I20260807 19:42:12.973145 2782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:12.974225 2782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:12.975466 2782 hybrid_clock.cc:648] HybridClock initialized: now 1786131732975403 us; error 54 us; skew 500 ppm
I20260807 19:42:12.975803 2782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:12 dist-test-slave-vq2q krb5kdc[2366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131732, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:12.979321 2782 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:42:12.980965 2782 webserver.cc:460] Webserver started at http://127.30.208.3:44931/ using document root <none> and password file <none>
I20260807 19:42:12.981735 2782 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:12.981849 2782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:12.982120 2782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:12.985368 2782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "f2606e57a84242d9a16d45975e91750d"
format_stamp: "Formatted at 2026-08-07 19:42:12 on dist-test-slave-vq2q"
server_key: "ae8646f3621de61e42ff7d3f08fb066b"
server_key_iv: "ee57256d8856815220f16983f927b39d"
server_key_version: "encryptionkey@0"
I20260807 19:42:12.986222 2782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "f2606e57a84242d9a16d45975e91750d"
format_stamp: "Formatted at 2026-08-07 19:42:12 on dist-test-slave-vq2q"
server_key: "ae8646f3621de61e42ff7d3f08fb066b"
server_key_iv: "ee57256d8856815220f16983f927b39d"
server_key_version: "encryptionkey@0"
I20260807 19:42:12.991729 2782 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:42:12.995349 2799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:12.996768 2782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:42:12.996939 2782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "f2606e57a84242d9a16d45975e91750d"
format_stamp: "Formatted at 2026-08-07 19:42:12 on dist-test-slave-vq2q"
server_key: "ae8646f3621de61e42ff7d3f08fb066b"
server_key_iv: "ee57256d8856815220f16983f927b39d"
server_key_version: "encryptionkey@0"
I20260807 19:42:12.997157 2782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:13.025025 2782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:13.029850 2782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:13.030194 2782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:13.031450 2782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:13.032861 2782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:13.032936 2782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:13.033030 2782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:13.033104 2782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:13.059963 2629 heartbeater.cc:503] Master 127.30.208.62:34155 was elected leader, sending a full tablet report...
I20260807 19:42:13.063170 2782 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:35685
I20260807 19:42:13.063230 2921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:35685 every 8 connection(s)
I20260807 19:42:13.064388 2782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:13.069157 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 2782
I20260807 19:42:13.069397 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:13.069968 31552 external_mini_cluster.cc:1428] Setting key 84ac6cd94837cc3468d5571522d12c41
Aug 07 19:42:13 dist-test-slave-vq2q krb5kdc[2366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131732, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:13.086596 2925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:13.066968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:50311 (local address 127.30.208.62:34155)
0807 19:42:13.067427 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:13.067433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:13.068358 (+ 925us) server_negotiation.cc:407] Connection header received
0807 19:42:13.070919 (+ 2561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:13.070925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:13.071015 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:13.071113 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:13.073397 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:13.074266 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:13.075799 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:13.076271 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:13.076402 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:13.080081 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:13.080088 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:13.080093 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:13.080137 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:13.083107 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:13.083917 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:13.083922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:13.083926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:13.084094 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:13.084604 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:13.084608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:13.084611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:13.084997 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:13.085152 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:13.086081 (+ 929us) server_negotiation.cc:299] Negotiation successful
0807 19:42:13.086303 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":219,"threads_started":1}
I20260807 19:42:13.087626 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:13.067510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34155 (local address 127.30.208.3:50311)
0807 19:42:13.068195 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:13.068237 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:13.070510 (+ 2273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:13.071466 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:13.071481 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:13.072250 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:13.073205 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:13.073237 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:13.074409 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:13.074414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:13.075664 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:13.075682 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:13.076504 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:13.076509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:13.076713 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:13.077423 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:13.077448 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:13.079836 (+ 2388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:13.083257 (+ 3421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:13.083265 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:13.083280 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:13.083777 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:13.084208 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:13.084212 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:13.084216 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:13.084470 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:13.085125 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:13.085132 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:13.085606 (+ 474us) client_negotiation.cc:769] Sending connection context
0807 19:42:13.085959 (+ 353us) client_negotiation.cc:240] Negotiation successful
0807 19:42:13.086449 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":272,"threads_started":1}
I20260807 19:42:13.088951 2923 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34155
I20260807 19:42:13.089274 2923 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:13.090001 2923 heartbeater.cc:511] Master 127.30.208.62:34155 requested a full tablet report, sending...
I20260807 19:42:13.091348 2414 ts_manager.cc:194] Registered new tserver with Master: f2606e57a84242d9a16d45975e91750d (127.30.208.3:35685)
I20260807 19:42:13.092133 2414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:50311
I20260807 19:42:13.098417 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:42:13 dist-test-slave-vq2q krb5kdc[2366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131733, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 19:42:13 dist-test-slave-vq2q krb5kdc[2366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131733, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:13.150593 2925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:13.137100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54112 (local address 127.30.208.62:34155)
0807 19:42:13.137304 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:13.137308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:13.137881 (+ 573us) server_negotiation.cc:407] Connection header received
0807 19:42:13.138058 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:13.138062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:13.138145 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:13.138227 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:13.139664 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:13.140376 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:13.141619 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:13.142052 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:13.142145 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:13.145825 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:13.145830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:13.145833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:13.145890 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:13.148329 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:13.148958 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:13.148962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:13.148965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:13.149094 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:13.149457 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:13.149460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:13.149462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:13.149791 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:13.150009 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:13.150199 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 19:42:13.150372 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 19:42:13.151350 2414 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54112
I20260807 19:42:13.153505 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 2483
Aug 07 19:42:13 dist-test-slave-vq2q krb5kdc[2366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131733, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:42:13.169498 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:13.154562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53680 (local address 127.30.208.1:38949)
0807 19:42:13.155156 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:13.155164 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:13.155189 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:42:13.155246 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:13.155252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:13.155469 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:13.155593 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:13.156895 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:13.158018 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:13.159390 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:13.160000 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:13.160103 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:13.164078 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:13.164092 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:13.164107 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:13.164154 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:13.166984 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:13.167675 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:13.167681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:13.167683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:13.167809 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:13.168123 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:13.168126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:13.168128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:13.168463 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:13.168696 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:13.168906 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 19:42:13.169107 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":309,"threads_started":1}
W20260807 19:42:13.170723 2624 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53680
W20260807 19:42:13.171819 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 2483: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:42:13.172035 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 2483
I20260807 19:42:13.191825 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 2636
Aug 07 19:42:13 dist-test-slave-vq2q krb5kdc[2366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131733, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:42:13.208736 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:13.192752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36242 (local address 127.30.208.2:37651)
0807 19:42:13.193309 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:13.193316 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:13.193344 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:42:13.193407 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:13.193413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:13.193632 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:13.193764 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:13.195081 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:13.196112 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:13.197544 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:13.198100 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:13.198205 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:13.202538 (+ 4333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:13.202554 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:13.202570 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:13.202609 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:13.205684 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:13.206394 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:13.206403 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:13.206405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:13.206552 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:13.207042 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:13.207047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:13.207049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:13.207452 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:13.207724 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:13.208105 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 19:42:13.208332 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":262,"threads_started":1}
W20260807 19:42:13.210187 2770 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36242
W20260807 19:42:13.216629 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 2636: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:42:13.216847 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 2636
I20260807 19:42:13.233053 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 2782
Aug 07 19:42:13 dist-test-slave-vq2q krb5kdc[2366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131733, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:42:13.249466 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:13.234147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45410 (local address 127.30.208.3:35685)
0807 19:42:13.234857 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:13.234864 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:13.234895 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:42:13.234982 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:13.234989 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:13.235202 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:13.235322 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:13.236651 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:13.237815 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:13.239047 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:13.239592 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:13.239694 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:13.243899 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:13.243944 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:13.243960 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:13.243999 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:13.246732 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:13.247411 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:13.247418 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:13.247421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:13.247556 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:13.247904 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:13.247907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:13.247909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:13.248276 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:13.248535 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:13.248813 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 19:42:13.249050 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":341,"threads_started":1}
W20260807 19:42:13.250733 2916 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45410
W20260807 19:42:13.251856 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 2782: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:42:13.252030 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 2782
I20260807 19:42:13.268352 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 2379
I20260807 19:42:13.269594 2448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:13.374792 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 2379
2026-08-07T19:42:13Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3718 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:13 dist-test-slave-vq2q krb5kdc[2942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:13 dist-test-slave-vq2q krb5kdc[2942](info): set up 2 sockets
Aug 07 19:42:13 dist-test-slave-vq2q krb5kdc[2942](info): commencing operation
krb5kdc: starting...
W20260807 19:42:14.428257 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:42:14 dist-test-slave-vq2q krb5kdc[2942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:42:14.645138 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:38419
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:38419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:14.970513 2955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:14.970805 2955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:14.970937 2955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:14.978168 2955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:14.978279 2955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:14.978334 2955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:14.978416 2955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:14.978479 2955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:14.988337 2955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:38419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:38419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:14.989688 2955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:14.991012 2955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:15.002435 2961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:15.002470 2960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:15.002523 2963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:15.003340 2955 server_base.cc:1034] running on GCE node
I20260807 19:42:15.004132 2955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:15.005606 2955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:15.006842 2955 hybrid_clock.cc:648] HybridClock initialized: now 1786131735006762 us; error 47 us; skew 500 ppm
I20260807 19:42:15.007185 2955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:15 dist-test-slave-vq2q krb5kdc[2942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131735, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:15.015817 2955 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:42:15.017709 2955 webserver.cc:460] Webserver started at http://127.30.208.62:42803/ using document root <none> and password file <none>
I20260807 19:42:15.018461 2955 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:15.018590 2955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:15.018916 2955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:15.022203 2955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "f41072242c7e4b0eb1bc1922719ce736"
format_stamp: "Formatted at 2026-08-07 19:42:15 on dist-test-slave-vq2q"
server_key: "7d2e0cc87b68a61b28c6457826c9955e"
server_key_iv: "2ad2bc8837b748e55c23ad8a52ef93d8"
server_key_version: "encryptionkey@0"
I20260807 19:42:15.023030 2955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "f41072242c7e4b0eb1bc1922719ce736"
format_stamp: "Formatted at 2026-08-07 19:42:15 on dist-test-slave-vq2q"
server_key: "7d2e0cc87b68a61b28c6457826c9955e"
server_key_iv: "2ad2bc8837b748e55c23ad8a52ef93d8"
server_key_version: "encryptionkey@0"
I20260807 19:42:15.028481 2955 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:42:15.032337 2970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:15.033766 2955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:42:15.033954 2955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "f41072242c7e4b0eb1bc1922719ce736"
format_stamp: "Formatted at 2026-08-07 19:42:15 on dist-test-slave-vq2q"
server_key: "7d2e0cc87b68a61b28c6457826c9955e"
server_key_iv: "2ad2bc8837b748e55c23ad8a52ef93d8"
server_key_version: "encryptionkey@0"
I20260807 19:42:15.034157 2955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:15.058820 2955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:15.063052 2955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:15.063328 2955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:15.081425 2955 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:38419
I20260807 19:42:15.081446 3024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:38419 every 8 connection(s)
I20260807 19:42:15.082791 2955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:15.086395 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 2955
I20260807 19:42:15.086606 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:42:15.086930 3025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:15.087414 31552 external_mini_cluster.cc:1428] Setting key 570426e251428c3102ec6f520ce3bf74
I20260807 19:42:15.101565 3025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736: Bootstrap starting.
Aug 07 19:42:15 dist-test-slave-vq2q krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:15.104698 3025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:15.106138 3025 log.cc:824] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:15.109520 3025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736: No bootstrap required, opened a new log
I20260807 19:42:15.113014 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:15.090005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34222 (local address 127.30.208.62:38419)
0807 19:42:15.090828 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:15.090843 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:15.090898 (+ 55us) server_negotiation.cc:407] Connection header received
0807 19:42:15.093741 (+ 2843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:15.093772 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:15.094302 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:15.094758 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:15.096131 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:15.098167 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:15.099462 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:15.100513 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:15.100690 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:15.104593 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:15.104605 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:15.104625 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:15.104675 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:15.108275 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:15.108976 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:15.108985 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:15.108993 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:15.109238 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:15.109690 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:15.109695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:15.109698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:15.110434 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:15.110711 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:15.111125 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 19:42:15.111583 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":332,"threads_started":1}
I20260807 19:42:15.118737 3025 raft_consensus.cc:348] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41072242c7e4b0eb1bc1922719ce736" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 38419 } }
I20260807 19:42:15.119199 3025 raft_consensus.cc:374] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:15.119304 3025 raft_consensus.cc:729] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41072242c7e4b0eb1bc1922719ce736, State: Initialized, Role: FOLLOWER
I20260807 19:42:15.119915 3025 consensus_queue.cc:260] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41072242c7e4b0eb1bc1922719ce736" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 38419 } }
I20260807 19:42:15.120133 3025 raft_consensus.cc:388] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:15.120232 3025 raft_consensus.cc:482] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:15.120347 3025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:15.122524 3025 raft_consensus.cc:504] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41072242c7e4b0eb1bc1922719ce736" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 38419 } }
I20260807 19:42:15.123064 3025 leader_election.cc:302] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f41072242c7e4b0eb1bc1922719ce736; no voters:
I20260807 19:42:15.123765 3025 leader_election.cc:288] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:15.123908 3031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:15.124547 3031 raft_consensus.cc:686] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [term 1 LEADER]: Becoming Leader. State: Replica: f41072242c7e4b0eb1bc1922719ce736, State: Running, Role: LEADER
I20260807 19:42:15.125147 3031 consensus_queue.cc:237] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41072242c7e4b0eb1bc1922719ce736" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 38419 } }
I20260807 19:42:15.125833 3025 sys_catalog.cc:564] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:15.128130 3032 sys_catalog.cc:455] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f41072242c7e4b0eb1bc1922719ce736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41072242c7e4b0eb1bc1922719ce736" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 38419 } } }
I20260807 19:42:15.128566 3032 sys_catalog.cc:458] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:15.129230 3033 sys_catalog.cc:455] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f41072242c7e4b0eb1bc1922719ce736. Latest consensus state: current_term: 1 leader_uuid: "f41072242c7e4b0eb1bc1922719ce736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f41072242c7e4b0eb1bc1922719ce736" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 38419 } } }
I20260807 19:42:15.129549 3033 sys_catalog.cc:458] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:15.130255 3038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:15.135037 3038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:15.144670 3038 catalog_manager.cc:1347] Generated new cluster ID: 41703c5516474d688bca1ec11bf66009
I20260807 19:42:15.144788 3038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:15.158821 3038 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:15.160123 3038 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:15.166865 3038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f41072242c7e4b0eb1bc1922719ce736: Generated new TSK 0
I20260807 19:42:15.167681 3038 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:42:15.222975 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:38419
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:15.556490 3054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:15.556780 3054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:15.556911 3054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:15.564457 3054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:15.564587 3054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:15.564738 3054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:42:15.574935 3054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:38419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:15.576251 3054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:15.577627 3054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:15.590603 3061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:15.590620 3062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:15.590795 3064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:15.591411 3054 server_base.cc:1034] running on GCE node
I20260807 19:42:15.591998 3054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:15.592912 3054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:15.594200 3054 hybrid_clock.cc:648] HybridClock initialized: now 1786131735594100 us; error 92 us; skew 500 ppm
I20260807 19:42:15.594501 3054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:15 dist-test-slave-vq2q krb5kdc[2942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131735, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:15.597673 3054 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:42:15.599207 3054 webserver.cc:460] Webserver started at http://127.30.208.1:37415/ using document root <none> and password file <none>
I20260807 19:42:15.599865 3054 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:15.599989 3054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:15.600282 3054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:15.603440 3054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "570d12cd272d48a5b32edf8d247af7c6"
format_stamp: "Formatted at 2026-08-07 19:42:15 on dist-test-slave-vq2q"
server_key: "b94f73beae4822dca3bfa72373cd8b3f"
server_key_iv: "ec86d95cf759f0cd5b071132f123d386"
server_key_version: "encryptionkey@0"
I20260807 19:42:15.604212 3054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "570d12cd272d48a5b32edf8d247af7c6"
format_stamp: "Formatted at 2026-08-07 19:42:15 on dist-test-slave-vq2q"
server_key: "b94f73beae4822dca3bfa72373cd8b3f"
server_key_iv: "ec86d95cf759f0cd5b071132f123d386"
server_key_version: "encryptionkey@0"
I20260807 19:42:15.609400 3054 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 19:42:15.612936 3072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:15.614382 3054 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:15.614547 3054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "570d12cd272d48a5b32edf8d247af7c6"
format_stamp: "Formatted at 2026-08-07 19:42:15 on dist-test-slave-vq2q"
server_key: "b94f73beae4822dca3bfa72373cd8b3f"
server_key_iv: "ec86d95cf759f0cd5b071132f123d386"
server_key_version: "encryptionkey@0"
I20260807 19:42:15.614763 3054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:15.643857 3054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:15.648113 3054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:15.648370 3054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:15.649335 3054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:15.650645 3054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:15.650717 3054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:15.650816 3054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:15.650861 3054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:15.682013 3054 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:44907
I20260807 19:42:15.682065 3196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:44907 every 8 connection(s)
I20260807 19:42:15.683324 3054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:15.685022 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 3054
I20260807 19:42:15.685251 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:15.685977 31552 external_mini_cluster.cc:1428] Setting key 93655994846208f689958d0959e7a115
Aug 07 19:42:15 dist-test-slave-vq2q krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131735, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:15.705452 3199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:15.686152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:55899 (local address 127.30.208.62:38419)
0807 19:42:15.686619 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:15.686625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:15.687681 (+ 1056us) server_negotiation.cc:407] Connection header received
0807 19:42:15.689652 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:15.689657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:15.689745 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:15.689862 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:15.691496 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:15.692451 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:15.694210 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:15.694765 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:15.694884 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:15.698826 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:15.698833 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:15.698837 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:15.698881 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:15.701960 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:15.702810 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:15.702816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:15.702820 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:15.702979 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:15.703427 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:15.703432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:15.703435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:15.703790 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:15.703960 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:15.704821 (+ 861us) server_negotiation.cc:299] Negotiation successful
0807 19:42:15.705061 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":211,"threads_started":1}
I20260807 19:42:15.706254 3201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:15.686729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:38419 (local address 127.30.208.1:55899)
0807 19:42:15.687523 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:15.687577 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:15.689385 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:15.690126 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:15.690136 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:15.690681 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:15.691301 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:15.691320 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:15.692648 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:15.692652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:15.694054 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:15.694068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:15.695041 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:15.695048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:15.695336 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:15.696112 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:15.696144 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:15.698504 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:15.702133 (+ 3629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:15.702141 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:15.702156 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:15.702666 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:15.703085 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:15.703088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:15.703091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:15.703313 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:15.703977 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:15.703986 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:15.704378 (+ 392us) client_negotiation.cc:769] Sending connection context
0807 19:42:15.704630 (+ 252us) client_negotiation.cc:240] Negotiation successful
0807 19:42:15.705056 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":312,"threads_started":1}
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
I20260807 19:42:15.707849 3197 heartbeater.cc:347] Connected to a master server at 127.30.208.62:38419
I20260807 19:42:15.708213 3197 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:15.708969 3197 heartbeater.cc:511] Master 127.30.208.62:38419 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
I20260807 19:42:15.711321 2988 ts_manager.cc:194] Registered new tserver with Master: 570d12cd272d48a5b32edf8d247af7c6 (127.30.208.1:44907)
I20260807 19:42:15.713300 2988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:55899
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:42:15.736646 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:38419
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:16.067433 3205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:16.067723 3205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:16.067828 3205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:16.075189 3205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:16.075301 3205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:16.075429 3205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:42:16.085386 3205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:38419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:16.086707 3205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:16.087976 3205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:16.102145 3210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:16.102141 3211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:16.102088 3213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:16.102918 3205 server_base.cc:1034] running on GCE node
I20260807 19:42:16.103509 3205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:16.104445 3205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:16.105651 3205 hybrid_clock.cc:648] HybridClock initialized: now 1786131736105583 us; error 54 us; skew 500 ppm
I20260807 19:42:16.105984 3205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:16 dist-test-slave-vq2q krb5kdc[2942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131736, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:16.109122 3205 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:42:16.110569 3205 webserver.cc:460] Webserver started at http://127.30.208.2:45339/ using document root <none> and password file <none>
I20260807 19:42:16.111223 3205 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:16.111340 3205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:16.111655 3205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:16.114712 3205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "df4d8aa7866e4bd59ff4b9e7aa832613"
format_stamp: "Formatted at 2026-08-07 19:42:16 on dist-test-slave-vq2q"
server_key: "63b62849deb55643152b6169fc0ea14b"
server_key_iv: "4ab5739a9323fa2b0c16df70098ebc4f"
server_key_version: "encryptionkey@0"
I20260807 19:42:16.115482 3205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "df4d8aa7866e4bd59ff4b9e7aa832613"
format_stamp: "Formatted at 2026-08-07 19:42:16 on dist-test-slave-vq2q"
server_key: "63b62849deb55643152b6169fc0ea14b"
server_key_iv: "4ab5739a9323fa2b0c16df70098ebc4f"
server_key_version: "encryptionkey@0"
I20260807 19:42:16.120716 3205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 19:42:16.124131 3220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:16.125502 3205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:42:16.125676 3205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "df4d8aa7866e4bd59ff4b9e7aa832613"
format_stamp: "Formatted at 2026-08-07 19:42:16 on dist-test-slave-vq2q"
server_key: "63b62849deb55643152b6169fc0ea14b"
server_key_iv: "4ab5739a9323fa2b0c16df70098ebc4f"
server_key_version: "encryptionkey@0"
I20260807 19:42:16.125890 3205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:16.141692 3205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:16.145808 3205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:16.146083 3205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:16.147059 3205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:16.148430 3205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:16.148502 3205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:16.148615 3205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:16.148671 3205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:16.180166 3205 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:42859
I20260807 19:42:16.180212 3344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:42859 every 8 connection(s)
I20260807 19:42:16.181527 3205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:16.188704 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 3205
I20260807 19:42:16.188890 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:16.189389 31552 external_mini_cluster.cc:1428] Setting key 499c0263f49f7c693f014b43d6248b61
Aug 07 19:42:16 dist-test-slave-vq2q krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131736, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:16.201673 3199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:16.183957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:56481 (local address 127.30.208.62:38419)
0807 19:42:16.184179 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:16.184185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:16.185384 (+ 1199us) server_negotiation.cc:407] Connection header received
0807 19:42:16.187196 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:16.187200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:16.187276 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:16.187366 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:16.189060 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.189860 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.191730 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.192150 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.192273 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:16.195920 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:16.195926 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:16.195929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:16.195963 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:16.198440 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.199213 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.199220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.199222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.199417 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.199949 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.199952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.199954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.200227 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:16.200349 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:16.201225 (+ 876us) server_negotiation.cc:299] Negotiation successful
0807 19:42:16.201418 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:42:16.203068 3347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:16.184438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:38419 (local address 127.30.208.2:56481)
0807 19:42:16.185237 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:16.185275 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:16.186935 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:16.187558 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:16.187566 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:16.188106 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:16.188872 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:16.188899 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.190030 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.190035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:16.191589 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:16.191602 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.192374 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.192382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:16.192624 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:16.193381 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:16.193405 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:16.195657 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:16.198582 (+ 2925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:16.198593 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:16.198611 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:16.199044 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:16.199558 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:16.199563 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:16.199566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:16.199837 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:16.200355 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:16.200363 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:16.200803 (+ 440us) client_negotiation.cc:769] Sending connection context
0807 19:42:16.201108 (+ 305us) client_negotiation.cc:240] Negotiation successful
0807 19:42:16.201720 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":284,"threads_started":1}
I20260807 19:42:16.204571 3345 heartbeater.cc:347] Connected to a master server at 127.30.208.62:38419
I20260807 19:42:16.204866 3345 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:16.205631 3345 heartbeater.cc:511] Master 127.30.208.62:38419 requested a full tablet report, sending...
I20260807 19:42:16.207031 2988 ts_manager.cc:194] Registered new tserver with Master: df4d8aa7866e4bd59ff4b9e7aa832613 (127.30.208.2:42859)
I20260807 19:42:16.207835 2988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:56481
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:42:16.240984 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:38419
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:16.570914 3350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:16.571225 3350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:16.571358 3350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:16.578712 3350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:16.578858 3350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:16.579010 3350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:42:16.588774 3350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:38419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:16.590113 3350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:16.591439 3350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:16.605046 3355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:16.605074 3356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:16.605000 3358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:16.606316 3350 server_base.cc:1034] running on GCE node
I20260807 19:42:16.606948 3350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:16.607959 3350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:16.609161 3350 hybrid_clock.cc:648] HybridClock initialized: now 1786131736609116 us; error 40 us; skew 500 ppm
I20260807 19:42:16.609522 3350 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:16 dist-test-slave-vq2q krb5kdc[2942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131736, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:16.612571 3350 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:42:16.614219 3350 webserver.cc:460] Webserver started at http://127.30.208.3:36873/ using document root <none> and password file <none>
I20260807 19:42:16.614929 3350 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:16.615056 3350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:16.615357 3350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:16.618467 3350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "0cc038f95fcc45e8b6e97425be5eafb8"
format_stamp: "Formatted at 2026-08-07 19:42:16 on dist-test-slave-vq2q"
server_key: "64e6c13c7bca1f16e028055cb90f6252"
server_key_iv: "640427ed81d78302e333eb47bbee1c31"
server_key_version: "encryptionkey@0"
I20260807 19:42:16.619308 3350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "0cc038f95fcc45e8b6e97425be5eafb8"
format_stamp: "Formatted at 2026-08-07 19:42:16 on dist-test-slave-vq2q"
server_key: "64e6c13c7bca1f16e028055cb90f6252"
server_key_iv: "640427ed81d78302e333eb47bbee1c31"
server_key_version: "encryptionkey@0"
I20260807 19:42:16.624761 3350 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 19:42:16.628422 3368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:16.629868 3350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:16.630031 3350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "0cc038f95fcc45e8b6e97425be5eafb8"
format_stamp: "Formatted at 2026-08-07 19:42:16 on dist-test-slave-vq2q"
server_key: "64e6c13c7bca1f16e028055cb90f6252"
server_key_iv: "640427ed81d78302e333eb47bbee1c31"
server_key_version: "encryptionkey@0"
I20260807 19:42:16.630221 3350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:16.661104 3350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:16.665354 3350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:16.665612 3350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:16.666574 3350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:16.667927 3350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:16.668002 3350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:16.668092 3350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:16.668144 3350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:16.704183 3350 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:34517
I20260807 19:42:16.704263 3491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:34517 every 8 connection(s)
I20260807 19:42:16.705565 3350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:16.713429 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 3350
I20260807 19:42:16.713605 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:16.714038 31552 external_mini_cluster.cc:1428] Setting key 4ecceb1651e0353cca022f7693254878
I20260807 19:42:16.716737 3197 heartbeater.cc:503] Master 127.30.208.62:38419 was elected leader, sending a full tablet report...
Aug 07 19:42:16 dist-test-slave-vq2q krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131736, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:16.726604 3494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:16.708182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:34383 (local address 127.30.208.62:38419)
0807 19:42:16.708686 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:16.708691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:16.709533 (+ 842us) server_negotiation.cc:407] Connection header received
0807 19:42:16.711441 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:16.711448 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:16.711537 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:16.711633 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:16.713487 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.714422 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.716389 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.716878 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.716989 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:16.720783 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:16.720789 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:16.720792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:16.720824 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:16.723159 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.723916 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.723921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.723923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.724135 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.724677 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.724681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.724684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.725013 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:16.725155 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:16.725982 (+ 827us) server_negotiation.cc:299] Negotiation successful
0807 19:42:16.726227 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":284,"threads_started":1}
I20260807 19:42:16.727602 3495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:16.708629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:38419 (local address 127.30.208.3:34383)
0807 19:42:16.709372 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:16.709411 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:16.711121 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:16.711892 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:16.711901 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:16.712514 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:16.713250 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:16.713288 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.714604 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.714609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:16.716244 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:16.716257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.717103 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.717110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:16.717438 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:16.718235 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:16.718266 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:16.720565 (+ 2299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:16.723296 (+ 2731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:16.723306 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:16.723324 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:16.723777 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:16.724299 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:16.724304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:16.724308 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:16.724552 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:16.725146 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:16.725154 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:16.725551 (+ 397us) client_negotiation.cc:769] Sending connection context
0807 19:42:16.725816 (+ 265us) client_negotiation.cc:240] Negotiation successful
0807 19:42:16.726385 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":277,"threads_started":1}
I20260807 19:42:16.729097 3492 heartbeater.cc:347] Connected to a master server at 127.30.208.62:38419
I20260807 19:42:16.729506 3492 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:16.730353 3492 heartbeater.cc:511] Master 127.30.208.62:38419 requested a full tablet report, sending...
I20260807 19:42:16.731776 2988 ts_manager.cc:194] Registered new tserver with Master: 0cc038f95fcc45e8b6e97425be5eafb8 (127.30.208.3:34517)
I20260807 19:42:16.732592 2988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:34383
I20260807 19:42:16.742065 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:42:16 dist-test-slave-vq2q krb5kdc[2942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:42:16 dist-test-slave-vq2q krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:42:16.794435 3500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:16.779706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57690 (local address 127.30.208.1:44907)
0807 19:42:16.780458 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:16.780466 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:16.780495 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:42:16.780563 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:16.780569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:16.780777 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:16.780932 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:16.782319 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.783351 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.784550 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.785137 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.785269 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:16.789145 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:16.789157 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:16.789201 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:16.789247 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:16.791850 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.792448 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.792455 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.792457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.792581 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.792926 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.792929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.792931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.793314 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:16.793551 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:16.793849 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 19:42:16.794072 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":440,"threads_started":1}
Aug 07 19:42:16 dist-test-slave-vq2q krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:16.812345 3494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:16.798707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34238 (local address 127.30.208.62:38419)
0807 19:42:16.798952 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:16.798959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:16.799124 (+ 165us) server_negotiation.cc:407] Connection header received
0807 19:42:16.799315 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:16.799319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:16.799401 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:16.799486 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:16.800920 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.801698 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.802923 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.803294 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.803380 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:16.807196 (+ 3816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:16.807202 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:16.807205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:16.807239 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:16.809713 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.810351 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.810356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.810358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.810507 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.810958 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.810962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.810964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.811306 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:16.811522 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:16.811827 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 19:42:16.812030 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260807 19:42:16.813032 2988 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34238
I20260807 19:42:16.814100 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 3054
I20260807 19:42:16.827387 3500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:16.815117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57706 (local address 127.30.208.1:44907)
0807 19:42:16.815377 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:16.815384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:16.815525 (+ 141us) server_negotiation.cc:407] Connection header received
0807 19:42:16.815696 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:16.815701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:16.815807 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:16.815896 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:16.817146 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.817912 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.819183 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.819619 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.819730 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:16.822084 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:16.822089 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:16.822092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:16.822125 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:16.824703 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.825441 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.825446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.825448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.825578 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.825959 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.825963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.825965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.826265 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:16.826501 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:16.826821 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 19:42:16.827062 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 19:42:16.828065 3191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:16.911113 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 3054
I20260807 19:42:16.929399 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 3205
Aug 07 19:42:16 dist-test-slave-vq2q krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:42:16.946372 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:16.930385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48100 (local address 127.30.208.2:42859)
0807 19:42:16.930968 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:16.930976 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:16.931002 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:42:16.931085 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:16.931092 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:16.931307 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:16.931447 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:16.932594 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.933772 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.935237 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:16.935877 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:16.936020 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:16.940398 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:16.940420 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:16.940443 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:16.940498 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:16.943538 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.944309 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.944317 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.944320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.944449 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:16.944862 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:16.944865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:16.944866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:16.945274 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:16.945527 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:16.945809 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 19:42:16.946061 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":317,"threads_started":1}
I20260807 19:42:16.947757 3339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:17.031256 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 3205
I20260807 19:42:17.047955 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 3350
Aug 07 19:42:17 dist-test-slave-vq2q krb5kdc[2942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:42:17.064435 3507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:17.048934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35062 (local address 127.30.208.3:34517)
0807 19:42:17.049573 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:17.049580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:17.049609 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:42:17.049671 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:17.049678 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:17.049890 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:17.050035 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:17.051387 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:17.052403 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:17.053809 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:17.054383 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:17.054509 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:17.058901 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:17.058915 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:17.058930 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:17.058968 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:17.061861 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:17.062562 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:17.062569 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:17.062571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:17.062691 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:17.062968 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:17.062971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:17.062973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:17.063326 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:17.063581 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:17.063892 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 19:42:17.064118 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":344,"threads_started":1}
I20260807 19:42:17.065788 3486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:17.149499 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 3350
I20260807 19:42:17.165470 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 2955
I20260807 19:42:17.166839 3020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:17.272238 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 2955
2026-08-07T19:42:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3898 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:17 dist-test-slave-vq2q krb5kdc[3512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:17 dist-test-slave-vq2q krb5kdc[3512](info): set up 2 sockets
Aug 07 19:42:17 dist-test-slave-vq2q krb5kdc[3512](info): commencing operation
krb5kdc: starting...
W20260807 19:42:18.363296 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:42:18 dist-test-slave-vq2q krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:42:18.585275 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:42743
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:42743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:18.910296 3527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:18.910593 3527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:18.910732 3527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:18.918078 3527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:18.918193 3527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:18.918246 3527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:18.918324 3527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:18.918386 3527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:18.928416 3527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:42743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:42743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:18.929791 3527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:18.931118 3527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:18.942579 3532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:18.942538 3533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:18.942615 3535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:18.943512 3527 server_base.cc:1034] running on GCE node
I20260807 19:42:18.944348 3527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:18.945858 3527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:18.947089 3527 hybrid_clock.cc:648] HybridClock initialized: now 1786131738947031 us; error 48 us; skew 500 ppm
I20260807 19:42:18.947451 3527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:18 dist-test-slave-vq2q krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131738, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:18.950567 3527 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:42:18.951747 3527 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:18.951889 3527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:18.952201 3527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:18.955188 3527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "1ae3ea77d38d4635b610861f51bef397"
format_stamp: "Formatted at 2026-08-07 19:42:18 on dist-test-slave-vq2q"
server_key: "c7a3eb25bb8691bafc2bef251fd60ca5"
server_key_iv: "8556db0639e042b707f64e62ef4a15b0"
server_key_version: "encryptionkey@0"
I20260807 19:42:18.956008 3527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "1ae3ea77d38d4635b610861f51bef397"
format_stamp: "Formatted at 2026-08-07 19:42:18 on dist-test-slave-vq2q"
server_key: "c7a3eb25bb8691bafc2bef251fd60ca5"
server_key_iv: "8556db0639e042b707f64e62ef4a15b0"
server_key_version: "encryptionkey@0"
I20260807 19:42:18.961050 3527 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 19:42:18.964552 3541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:18.965881 3527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:18.966051 3527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "1ae3ea77d38d4635b610861f51bef397"
format_stamp: "Formatted at 2026-08-07 19:42:18 on dist-test-slave-vq2q"
server_key: "c7a3eb25bb8691bafc2bef251fd60ca5"
server_key_iv: "8556db0639e042b707f64e62ef4a15b0"
server_key_version: "encryptionkey@0"
I20260807 19:42:18.966252 3527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:18.995054 3527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:18.999600 3527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:18.999887 3527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:19.017894 3527 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:42743
I20260807 19:42:19.017913 3596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:42743 every 8 connection(s)
I20260807 19:42:19.018831 3527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:19.022523 3597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:19.026902 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 3527
I20260807 19:42:19.027120 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:42:19.027908 31552 external_mini_cluster.cc:1428] Setting key ed89c10f91acbb90d601c50f35fc268f
I20260807 19:42:19.034431 3597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397: Bootstrap starting.
I20260807 19:42:19.038043 3597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:19.039278 3597 log.cc:824] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:19.042737 3597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397: No bootstrap required, opened a new log
Aug 07 19:42:19 dist-test-slave-vq2q krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:19.051054 3597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae3ea77d38d4635b610861f51bef397" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 42743 } }
I20260807 19:42:19.051429 3597 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:19.051527 3597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ae3ea77d38d4635b610861f51bef397, State: Initialized, Role: FOLLOWER
I20260807 19:42:19.051966 3600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:19.030060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47816 (local address 127.30.208.62:42743)
0807 19:42:19.030870 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:19.030886 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:19.030932 (+ 46us) server_negotiation.cc:407] Connection header received
0807 19:42:19.032926 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:19.032951 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:19.033404 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:19.033726 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:19.035629 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:19.036989 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:19.038846 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:19.039577 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:19.039710 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:19.044522 (+ 4812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:19.044531 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:19.044545 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:19.044585 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:19.047721 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:19.048441 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:19.048449 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:19.048456 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:19.048650 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:19.049029 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:19.049033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:19.049035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:19.049683 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:19.049992 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:19.050457 (+ 465us) server_negotiation.cc:299] Negotiation successful
0807 19:42:19.050891 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":383,"threads_started":1}
I20260807 19:42:19.052162 3597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae3ea77d38d4635b610861f51bef397" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 42743 } }
I20260807 19:42:19.052459 3597 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:19.052606 3597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:19.052809 3597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:19.055707 3597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae3ea77d38d4635b610861f51bef397" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 42743 } }
I20260807 19:42:19.056150 3597 leader_election.cc:302] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ae3ea77d38d4635b610861f51bef397; no voters:
I20260807 19:42:19.056658 3597 leader_election.cc:288] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:19.056790 3602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:19.057929 3602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [term 1 LEADER]: Becoming Leader. State: Replica: 1ae3ea77d38d4635b610861f51bef397, State: Running, Role: LEADER
I20260807 19:42:19.058516 3602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae3ea77d38d4635b610861f51bef397" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 42743 } }
I20260807 19:42:19.058755 3597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:19.060910 3603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ae3ea77d38d4635b610861f51bef397" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae3ea77d38d4635b610861f51bef397" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 42743 } } }
I20260807 19:42:19.061247 3603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:19.061689 3604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ae3ea77d38d4635b610861f51bef397. Latest consensus state: current_term: 1 leader_uuid: "1ae3ea77d38d4635b610861f51bef397" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ae3ea77d38d4635b610861f51bef397" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 42743 } } }
I20260807 19:42:19.061997 3604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:19.062196 3608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:19.068817 3608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:19.077756 3608 catalog_manager.cc:1347] Generated new cluster ID: e3fa77aacbf0491882729dba97cc01f0
I20260807 19:42:19.077877 3608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:19.096105 3608 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:19.097527 3608 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:19.106994 3608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397: Generated new TSK 0
I20260807 19:42:19.107847 3608 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:42:19.168711 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:42743
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:19.512320 3625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:19.512648 3625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:19.512806 3625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:19.520256 3625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:19.520394 3625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:19.520567 3625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:42:19.531028 3625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:42743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:19.532516 3625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:19.533952 3625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:19.547245 3631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:19.547255 3630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:19.547271 3633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:19.548059 3625 server_base.cc:1034] running on GCE node
I20260807 19:42:19.548681 3625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:19.549726 3625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:19.550930 3625 hybrid_clock.cc:648] HybridClock initialized: now 1786131739550875 us; error 41 us; skew 500 ppm
I20260807 19:42:19.551257 3625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:19 dist-test-slave-vq2q krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131739, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:19.554385 3625 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:42:19.555482 3625 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:19.555572 3625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:19.555851 3625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:19.558809 3625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "e34072069e364e58b1ff988562f634c0"
format_stamp: "Formatted at 2026-08-07 19:42:19 on dist-test-slave-vq2q"
server_key: "3c641c36f55cd9e7360838b314ad184a"
server_key_iv: "52618dd1a232bd2f3f31e112e3c00b7a"
server_key_version: "encryptionkey@0"
I20260807 19:42:19.559614 3625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "e34072069e364e58b1ff988562f634c0"
format_stamp: "Formatted at 2026-08-07 19:42:19 on dist-test-slave-vq2q"
server_key: "3c641c36f55cd9e7360838b314ad184a"
server_key_iv: "52618dd1a232bd2f3f31e112e3c00b7a"
server_key_version: "encryptionkey@0"
I20260807 19:42:19.564746 3625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260807 19:42:19.568140 3639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:19.569492 3625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:19.569680 3625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "e34072069e364e58b1ff988562f634c0"
format_stamp: "Formatted at 2026-08-07 19:42:19 on dist-test-slave-vq2q"
server_key: "3c641c36f55cd9e7360838b314ad184a"
server_key_iv: "52618dd1a232bd2f3f31e112e3c00b7a"
server_key_version: "encryptionkey@0"
I20260807 19:42:19.569912 3625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:19.586174 3625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:19.590487 3625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:19.590729 3625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:19.591656 3625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:19.593019 3625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:19.593107 3625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:19.593215 3625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:19.593291 3625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 19:42:19.623075 3625 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:39771
I20260807 19:42:19.623144 3764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:39771 every 8 connection(s)
I20260807 19:42:19.623893 3625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:19.624795 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 3625
I20260807 19:42:19.625010 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:19.625890 31552 external_mini_cluster.cc:1428] Setting key 164e361cdf76f3cd1c2212993e873260
Aug 07 19:42:19 dist-test-slave-vq2q krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131739, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:19.645737 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:19.626610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:57527 (local address 127.30.208.62:42743)
0807 19:42:19.627096 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:19.627102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:19.628132 (+ 1030us) server_negotiation.cc:407] Connection header received
0807 19:42:19.629980 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:19.629985 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:19.630063 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:19.630158 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:19.631900 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:19.632664 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:19.634776 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:19.635190 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:19.635310 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:19.638779 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:19.638786 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:19.638791 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:19.638832 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:19.641939 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:19.642844 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:19.642850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:19.642854 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:19.643001 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:19.643485 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:19.643489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:19.643492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:19.643858 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:19.644012 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:19.645042 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0807 19:42:19.645332 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":237,"threads_started":1}
I20260807 19:42:19.646730 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:19.627190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:42743 (local address 127.30.208.1:57527)
0807 19:42:19.627982 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:19.628032 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:19.629719 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:19.630410 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:19.630418 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:19.630944 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:19.631677 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:19.631703 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:19.632842 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:19.632848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:19.634598 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:19.634615 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:19.635424 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:19.635430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:19.635631 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:19.636269 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:19.636298 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:19.638518 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:19.642117 (+ 3599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:19.642125 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:19.642140 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:19.642681 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:19.643134 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:19.643137 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:19.643140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:19.643358 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:19.643972 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:19.643980 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:19.644409 (+ 429us) client_negotiation.cc:769] Sending connection context
0807 19:42:19.644867 (+ 458us) client_negotiation.cc:240] Negotiation successful
0807 19:42:19.645516 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":308,"threads_started":1}
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
I20260807 19:42:19.648386 3765 heartbeater.cc:347] Connected to a master server at 127.30.208.62:42743
I20260807 19:42:19.648718 3765 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:19.649477 3765 heartbeater.cc:511] Master 127.30.208.62:42743 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
I20260807 19:42:19.651715 3559 ts_manager.cc:194] Registered new tserver with Master: e34072069e364e58b1ff988562f634c0 (127.30.208.1:39771)
I20260807 19:42:19.653762 3559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:57527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:42:19.680806 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:42743
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:20.011153 3771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:20.011446 3771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:20.011581 3771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:20.019760 3771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:20.019908 3771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:20.020082 3771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:42:20.030115 3771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:42743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:20.031407 3771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:20.032698 3771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:20.045329 3777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:20.045385 3780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:20.045356 3778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:20.046334 3771 server_base.cc:1034] running on GCE node
I20260807 19:42:20.046988 3771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:20.047973 3771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:20.049212 3771 hybrid_clock.cc:648] HybridClock initialized: now 1786131740049121 us; error 48 us; skew 500 ppm
I20260807 19:42:20.049532 3771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:20 dist-test-slave-vq2q krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:20.055706 3771 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:42:20.056874 3771 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:20.056985 3771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:20.057340 3771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:20.060537 3771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "c740b91a0f8f4f64b50853c4efaf86b9"
format_stamp: "Formatted at 2026-08-07 19:42:20 on dist-test-slave-vq2q"
server_key: "5d3da3693c94b3ce57f02de52b3baeaa"
server_key_iv: "f784601eb9f3df19ef206efd72db3b34"
server_key_version: "encryptionkey@0"
I20260807 19:42:20.061455 3771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "c740b91a0f8f4f64b50853c4efaf86b9"
format_stamp: "Formatted at 2026-08-07 19:42:20 on dist-test-slave-vq2q"
server_key: "5d3da3693c94b3ce57f02de52b3baeaa"
server_key_iv: "f784601eb9f3df19ef206efd72db3b34"
server_key_version: "encryptionkey@0"
I20260807 19:42:20.066834 3771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:42:20.070560 3786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:20.072016 3771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:20.072190 3771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "c740b91a0f8f4f64b50853c4efaf86b9"
format_stamp: "Formatted at 2026-08-07 19:42:20 on dist-test-slave-vq2q"
server_key: "5d3da3693c94b3ce57f02de52b3baeaa"
server_key_iv: "f784601eb9f3df19ef206efd72db3b34"
server_key_version: "encryptionkey@0"
I20260807 19:42:20.072412 3771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:20.085588 3771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:20.089838 3771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:20.090097 3771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:20.091063 3771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:20.092446 3771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:20.092547 3771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:20.092648 3771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:20.092717 3771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:20.125027 3771 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:35679
I20260807 19:42:20.125089 3905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:35679 every 8 connection(s)
I20260807 19:42:20.125926 3771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:20.132838 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 3771
I20260807 19:42:20.133028 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:20.133538 31552 external_mini_cluster.cc:1428] Setting key 7717894316be99e47dda07cf01118480
Aug 07 19:42:20 dist-test-slave-vq2q krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:20.147090 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.128477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:60717 (local address 127.30.208.62:42743)
0807 19:42:20.128689 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.128696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.129875 (+ 1179us) server_negotiation.cc:407] Connection header received
0807 19:42:20.131723 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.131736 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.131813 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.131908 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:20.133478 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.134409 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.136112 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.136724 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.136859 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.140802 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:20.140807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:20.140810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:20.140846 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:20.143803 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.144647 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.144653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.144656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.144808 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.145228 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.145233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.145236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.145576 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:20.145731 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.146486 (+ 755us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.146727 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:42:20.147889 3908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.128909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:42743 (local address 127.30.208.2:60717)
0807 19:42:20.129698 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:20.129738 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:20.131392 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:20.132105 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:20.132113 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:20.132633 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:20.133297 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.133318 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.134568 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.134572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.135958 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.135971 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.136951 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.136957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.137157 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.137845 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:20.137867 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:20.140518 (+ 2651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:20.143971 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:20.143978 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:20.143992 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:20.144512 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:20.144908 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:20.144912 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:20.144914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:20.145097 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:20.145696 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:20.145701 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:20.146042 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 19:42:20.146344 (+ 302us) client_negotiation.cc:240] Negotiation successful
0807 19:42:20.146747 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":342,"threads_started":1}
I20260807 19:42:20.149365 3906 heartbeater.cc:347] Connected to a master server at 127.30.208.62:42743
I20260807 19:42:20.149658 3906 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:20.150305 3906 heartbeater.cc:511] Master 127.30.208.62:42743 requested a full tablet report, sending...
I20260807 19:42:20.151684 3559 ts_manager.cc:194] Registered new tserver with Master: c740b91a0f8f4f64b50853c4efaf86b9 (127.30.208.2:35679)
I20260807 19:42:20.152483 3559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:60717
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:42:20.184525 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:42743
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:40071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:20.508718 3911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:20.509037 3911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:20.509169 3911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:20.516559 3911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:20.516693 3911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:20.516865 3911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:42:20.526830 3911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:40071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:42743
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:20.528187 3911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:20.529597 3911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:20.542281 3920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:20.542510 3916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:20.542541 3917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:20.543792 3911 server_base.cc:1034] running on GCE node
I20260807 19:42:20.544428 3911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:20.545415 3911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:20.546622 3911 hybrid_clock.cc:648] HybridClock initialized: now 1786131740546565 us; error 46 us; skew 500 ppm
I20260807 19:42:20.546981 3911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:20 dist-test-slave-vq2q krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:20.550225 3911 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:42:20.551501 3911 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:20.551610 3911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:20.551944 3911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:20.555089 3911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "f9fed0f029de4108ba37caa508f5b9d0"
format_stamp: "Formatted at 2026-08-07 19:42:20 on dist-test-slave-vq2q"
server_key: "caac1b3a632b75bd44df724dfbf582e4"
server_key_iv: "c856bb2a061297cbbb655384af681b59"
server_key_version: "encryptionkey@0"
I20260807 19:42:20.555918 3911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "f9fed0f029de4108ba37caa508f5b9d0"
format_stamp: "Formatted at 2026-08-07 19:42:20 on dist-test-slave-vq2q"
server_key: "caac1b3a632b75bd44df724dfbf582e4"
server_key_iv: "c856bb2a061297cbbb655384af681b59"
server_key_version: "encryptionkey@0"
I20260807 19:42:20.561369 3911 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:42:20.565248 3926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:20.566716 3911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:20.566907 3911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "f9fed0f029de4108ba37caa508f5b9d0"
format_stamp: "Formatted at 2026-08-07 19:42:20 on dist-test-slave-vq2q"
server_key: "caac1b3a632b75bd44df724dfbf582e4"
server_key_iv: "c856bb2a061297cbbb655384af681b59"
server_key_version: "encryptionkey@0"
I20260807 19:42:20.567134 3911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:20.586161 3911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:20.590200 3911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:20.590469 3911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:20.591481 3911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:20.592890 3911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:20.592988 3911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:20.593101 3911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:20.593190 3911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:20.625094 3911 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:41281
I20260807 19:42:20.625129 4049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:41281 every 8 connection(s)
I20260807 19:42:20.626310 3911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:20.635914 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 3911
I20260807 19:42:20.636099 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:20.636634 31552 external_mini_cluster.cc:1428] Setting key e086311049015f976ef55867d1dfa8ce
Aug 07 19:42:20 dist-test-slave-vq2q krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131740, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:20.646663 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.628968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:54411 (local address 127.30.208.62:42743)
0807 19:42:20.629132 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.629137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.630578 (+ 1441us) server_negotiation.cc:407] Connection header received
0807 19:42:20.632377 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.632382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.632457 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.632548 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:20.634049 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.634865 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.636919 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.637430 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.637544 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.641391 (+ 3847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:20.641396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:20.641400 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:20.641435 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:20.643938 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.644619 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.644623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.644625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.644754 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.645113 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.645116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.645118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.645423 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:20.645537 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.646215 (+ 678us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.646400 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:42:20.647651 4056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.629625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:42743 (local address 127.30.208.3:54411)
0807 19:42:20.630392 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:20.630438 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:20.632093 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:20.632726 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:20.632734 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:20.633283 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:20.633872 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.633890 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.635001 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.635005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.636745 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.636762 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.637658 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.637666 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.637874 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.638574 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:20.638598 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:20.641155 (+ 2557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:20.644070 (+ 2915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:20.644077 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:20.644090 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:20.644524 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:20.644856 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:20.644859 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:20.644861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:20.645032 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:20.645541 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:20.645547 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:20.645900 (+ 353us) client_negotiation.cc:769] Sending connection context
0807 19:42:20.646127 (+ 227us) client_negotiation.cc:240] Negotiation successful
0807 19:42:20.646514 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":323,"threads_started":1}
I20260807 19:42:20.649093 4050 heartbeater.cc:347] Connected to a master server at 127.30.208.62:42743
I20260807 19:42:20.649442 4050 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:20.650082 4050 heartbeater.cc:511] Master 127.30.208.62:42743 requested a full tablet report, sending...
I20260807 19:42:20.651793 3559 ts_manager.cc:194] Registered new tserver with Master: f9fed0f029de4108ba37caa508f5b9d0 (127.30.208.3:41281)
I20260807 19:42:20.652264 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:42:20.652644 3559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:54411
I20260807 19:42:20.656888 3765 heartbeater.cc:503] Master 127.30.208.62:42743 was elected leader, sending a full tablet report...
I20260807 19:42:20.668965 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.656835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58090 (local address 127.30.208.62:42743)
0807 19:42:20.657036 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.657043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.657551 (+ 508us) server_negotiation.cc:407] Connection header received
0807 19:42:20.657731 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.657740 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.657825 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.657917 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:20.659393 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.660135 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.661449 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.661866 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.661958 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.664272 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:20.664277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:20.664280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:20.664310 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:20.666791 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.667372 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.667376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.667378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.667500 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:20.667851 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:20.667854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:20.667856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:20.668156 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:20.668346 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.668582 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.668738 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:42:20.672962 3559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:20.680119 3559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:20.705897 4066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.692191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39771 (local address 127.0.0.1:58160)
0807 19:42:20.693027 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:20.693079 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:20.693304 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:20.698320 (+ 5016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:20.698324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:20.698386 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:20.699117 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.699129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.700935 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.700940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.703085 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.703095 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.704726 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.704733 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.704943 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.705014 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 19:42:20.705142 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 19:42:20.705380 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":134,"spinlock_wait_cycles":2944,"thread_start_us":438,"threads_started":1}
I20260807 19:42:20.706079 4070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.692357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:35679 (local address 127.0.0.1:48150)
0807 19:42:20.694030 (+ 1673us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:20.694048 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:20.694153 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:20.698306 (+ 4153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:20.698314 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:20.698355 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:20.698902 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.698918 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.700614 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.700618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.703043 (+ 2425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.703059 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.704689 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.704702 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.705310 (+ 608us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.705344 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:42:20.705429 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 19:42:20.705561 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1552,"mutex_wait_us":369,"spinlock_wait_cycles":1408,"thread_start_us":5381,"threads_started":1}
I20260807 19:42:20.706463 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.692436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58160 (local address 127.30.208.1:39771)
0807 19:42:20.697720 (+ 5284us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.697729 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.697761 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:42:20.697864 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.697872 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.698112 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.698772 (+ 660us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:20.699278 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.700804 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.703241 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.704441 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.705630 (+ 1189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.705895 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.705972 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.706152 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5108,"thread_start_us":4950,"threads_started":1}
I20260807 19:42:20.706964 4069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.693006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48150 (local address 127.30.208.2:35679)
0807 19:42:20.697599 (+ 4593us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.697609 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.697645 (+ 36us) server_negotiation.cc:407] Connection header received
0807 19:42:20.697757 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.697765 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.698033 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.699107 (+ 1074us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:20.699341 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.700483 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.703217 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.704428 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.704946 (+ 518us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.705159 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.706531 (+ 1372us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.706712 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4437,"thread_start_us":4333,"threads_started":1}
I20260807 19:42:20.707065 4067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.693510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:41281 (local address 127.0.0.1:49786)
0807 19:42:20.698021 (+ 4511us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:20.698039 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:20.698258 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:20.699097 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:20.699106 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:20.699130 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:20.699512 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.699525 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.701878 (+ 2353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.701884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.703636 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.703653 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.705754 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.705761 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.705936 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.705969 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:42:20.706310 (+ 341us) client_negotiation.cc:240] Negotiation successful
0807 19:42:20.706453 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4376,"mutex_wait_us":94,"thread_start_us":937,"threads_started":1}
I20260807 19:42:20.707623 4071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.694140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49786 (local address 127.30.208.3:41281)
0807 19:42:20.694940 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.694950 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.698149 (+ 3199us) server_negotiation.cc:407] Connection header received
0807 19:42:20.698436 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.698444 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.698650 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.698805 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:20.699752 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.701698 (+ 1946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.703847 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.705245 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.705367 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.705577 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.706525 (+ 948us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.707289 (+ 764us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":451,"threads_started":1}
I20260807 19:42:20.710646 3838 tablet_service.cc:1467] Processing CreateTablet for tablet d04ae63beb174bb49e79620d27c1d955 (DEFAULT_TABLE table=test-table [id=871cfeafd9dd40f48c0ec191654b1e09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:20.710646 3696 tablet_service.cc:1467] Processing CreateTablet for tablet d04ae63beb174bb49e79620d27c1d955 (DEFAULT_TABLE table=test-table [id=871cfeafd9dd40f48c0ec191654b1e09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:20.710875 3984 tablet_service.cc:1467] Processing CreateTablet for tablet d04ae63beb174bb49e79620d27c1d955 (DEFAULT_TABLE table=test-table [id=871cfeafd9dd40f48c0ec191654b1e09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:20.712051 3984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d04ae63beb174bb49e79620d27c1d955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:20.712111 3696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d04ae63beb174bb49e79620d27c1d955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:20.712111 3838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d04ae63beb174bb49e79620d27c1d955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:20.724275 4072 tablet_bootstrap.cc:492] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0: Bootstrap starting.
I20260807 19:42:20.727025 4073 tablet_bootstrap.cc:492] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9: Bootstrap starting.
I20260807 19:42:20.729781 4072 tablet_bootstrap.cc:654] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:20.729887 4074 tablet_bootstrap.cc:492] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0: Bootstrap starting.
I20260807 19:42:20.730963 4073 tablet_bootstrap.cc:654] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:20.731285 4072 log.cc:824] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:20.732175 4073 log.cc:824] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:20.732993 4074 tablet_bootstrap.cc:654] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:20.733935 4074 log.cc:824] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:20.734419 4072 tablet_bootstrap.cc:492] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0: No bootstrap required, opened a new log
I20260807 19:42:20.734715 4072 ts_tablet_manager.cc:1397] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 19:42:20.736435 4074 tablet_bootstrap.cc:492] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0: No bootstrap required, opened a new log
I20260807 19:42:20.736470 4073 tablet_bootstrap.cc:492] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9: No bootstrap required, opened a new log
I20260807 19:42:20.736788 4073 ts_tablet_manager.cc:1397] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260807 19:42:20.736804 4074 ts_tablet_manager.cc:1397] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 19:42:20.744163 4073 raft_consensus.cc:348] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34072069e364e58b1ff988562f634c0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39771 } } peers { permanent_uuid: "c740b91a0f8f4f64b50853c4efaf86b9" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 35679 } } peers { permanent_uuid: "f9fed0f029de4108ba37caa508f5b9d0" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41281 } }
I20260807 19:42:20.744551 4073 raft_consensus.cc:374] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:20.744663 4073 raft_consensus.cc:729] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c740b91a0f8f4f64b50853c4efaf86b9, State: Initialized, Role: FOLLOWER
I20260807 19:42:20.745323 4073 consensus_queue.cc:260] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34072069e364e58b1ff988562f634c0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39771 } } peers { permanent_uuid: "c740b91a0f8f4f64b50853c4efaf86b9" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 35679 } } peers { permanent_uuid: "f9fed0f029de4108ba37caa508f5b9d0" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41281 } }
I20260807 19:42:20.745932 4072 raft_consensus.cc:348] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34072069e364e58b1ff988562f634c0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39771 } } peers { permanent_uuid: "c740b91a0f8f4f64b50853c4efaf86b9" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 35679 } } peers { permanent_uuid: "f9fed0f029de4108ba37caa508f5b9d0" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41281 } }
I20260807 19:42:20.746394 4072 raft_consensus.cc:374] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:20.746474 3906 heartbeater.cc:503] Master 127.30.208.62:42743 was elected leader, sending a full tablet report...
I20260807 19:42:20.746522 4072 raft_consensus.cc:729] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9fed0f029de4108ba37caa508f5b9d0, State: Initialized, Role: FOLLOWER
I20260807 19:42:20.747376 4073 ts_tablet_manager.cc:1428] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9: Time spent starting tablet: real 0.010s user 0.004s sys 0.006s
I20260807 19:42:20.747224 4072 consensus_queue.cc:260] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34072069e364e58b1ff988562f634c0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39771 } } peers { permanent_uuid: "c740b91a0f8f4f64b50853c4efaf86b9" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 35679 } } peers { permanent_uuid: "f9fed0f029de4108ba37caa508f5b9d0" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41281 } }
I20260807 19:42:20.747291 4074 raft_consensus.cc:348] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34072069e364e58b1ff988562f634c0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39771 } } peers { permanent_uuid: "c740b91a0f8f4f64b50853c4efaf86b9" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 35679 } } peers { permanent_uuid: "f9fed0f029de4108ba37caa508f5b9d0" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41281 } }
I20260807 19:42:20.747716 4074 raft_consensus.cc:374] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:20.747839 4074 raft_consensus.cc:729] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e34072069e364e58b1ff988562f634c0, State: Initialized, Role: FOLLOWER
I20260807 19:42:20.748597 4074 consensus_queue.cc:260] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34072069e364e58b1ff988562f634c0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39771 } } peers { permanent_uuid: "c740b91a0f8f4f64b50853c4efaf86b9" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 35679 } } peers { permanent_uuid: "f9fed0f029de4108ba37caa508f5b9d0" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41281 } }
I20260807 19:42:20.749465 4050 heartbeater.cc:503] Master 127.30.208.62:42743 was elected leader, sending a full tablet report...
I20260807 19:42:20.751626 4072 ts_tablet_manager.cc:1428] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0: Time spent starting tablet: real 0.017s user 0.014s sys 0.000s
I20260807 19:42:20.752506 4074 ts_tablet_manager.cc:1428] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20260807 19:42:20.876070 3766 tablet.cc:2369] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:42:20.877919 3907 tablet.cc:2369] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:42:20.877893 4054 tablet.cc:2369] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:20.945484 4081 raft_consensus.cc:482] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:20.945776 4081 raft_consensus.cc:504] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34072069e364e58b1ff988562f634c0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39771 } } peers { permanent_uuid: "c740b91a0f8f4f64b50853c4efaf86b9" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 35679 } } peers { permanent_uuid: "f9fed0f029de4108ba37caa508f5b9d0" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41281 } }
I20260807 19:42:20.947510 4081 leader_election.cc:288] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c740b91a0f8f4f64b50853c4efaf86b9 (127.30.208.2:35679), f9fed0f029de4108ba37caa508f5b9d0 (127.30.208.3:41281)
I20260807 19:42:20.956332 4069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.947936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:55125 (local address 127.30.208.2:35679)
0807 19:42:20.948169 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.948176 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.948668 (+ 492us) server_negotiation.cc:407] Connection header received
0807 19:42:20.948873 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.948877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.948948 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.949045 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:20.949761 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.950931 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.953977 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.955334 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.955467 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.955530 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.955920 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.956060 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 19:42:20.956332 4071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.948145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:38899 (local address 127.30.208.3:41281)
0807 19:42:20.948369 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:20.948374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:20.948970 (+ 596us) server_negotiation.cc:407] Connection header received
0807 19:42:20.949114 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:20.949118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:20.949214 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:20.949287 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:20.949908 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.950989 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.953971 (+ 2982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.955256 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.955428 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.955507 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:20.955919 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 19:42:20.956060 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:42:20.956553 4082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.947820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:35679 (local address 127.30.208.1:55125)
0807 19:42:20.948540 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:20.948556 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:20.948732 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:20.949084 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:20.949089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:20.949113 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:20.949540 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.949555 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.951110 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.951116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.953789 (+ 2673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.953799 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.955531 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.955538 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.955758 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.955794 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:42:20.955865 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 19:42:20.956046 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"mutex_wait_us":118,"thread_start_us":366,"threads_started":1}
I20260807 19:42:20.956578 4083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.947836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:41281 (local address 127.30.208.1:38899)
0807 19:42:20.948863 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:20.948878 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:20.948991 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:20.949335 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:20.949339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:20.949362 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:20.949748 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.949757 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.951129 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.951133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.953767 (+ 2634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:20.953788 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:20.955477 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:20.955483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:20.955670 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:20.955761 (+ 91us) client_negotiation.cc:769] Sending connection context
0807 19:42:20.955858 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 19:42:20.956046 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"mutex_wait_us":212,"thread_start_us":696,"threads_started":1}
I20260807 19:42:20.957620 4004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d04ae63beb174bb49e79620d27c1d955" candidate_uuid: "e34072069e364e58b1ff988562f634c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9fed0f029de4108ba37caa508f5b9d0" is_pre_election: true
I20260807 19:42:20.957623 3859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d04ae63beb174bb49e79620d27c1d955" candidate_uuid: "e34072069e364e58b1ff988562f634c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c740b91a0f8f4f64b50853c4efaf86b9" is_pre_election: true
I20260807 19:42:20.958151 4004 raft_consensus.cc:2454] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e34072069e364e58b1ff988562f634c0 in term 0.
I20260807 19:42:20.958151 3859 raft_consensus.cc:2454] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e34072069e364e58b1ff988562f634c0 in term 0.
I20260807 19:42:20.958758 3645 leader_election.cc:302] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e34072069e364e58b1ff988562f634c0, f9fed0f029de4108ba37caa508f5b9d0; no voters:
I20260807 19:42:20.959190 4081 raft_consensus.cc:2781] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:20.959355 4081 raft_consensus.cc:482] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:20.959462 4081 raft_consensus.cc:3037] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:20.961720 4081 raft_consensus.cc:504] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34072069e364e58b1ff988562f634c0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39771 } } peers { permanent_uuid: "c740b91a0f8f4f64b50853c4efaf86b9" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 35679 } } peers { permanent_uuid: "f9fed0f029de4108ba37caa508f5b9d0" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41281 } }
I20260807 19:42:20.962373 4081 leader_election.cc:288] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [CANDIDATE]: Term 1 election: Requested vote from peers c740b91a0f8f4f64b50853c4efaf86b9 (127.30.208.2:35679), f9fed0f029de4108ba37caa508f5b9d0 (127.30.208.3:41281)
I20260807 19:42:20.962844 3859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d04ae63beb174bb49e79620d27c1d955" candidate_uuid: "e34072069e364e58b1ff988562f634c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c740b91a0f8f4f64b50853c4efaf86b9"
I20260807 19:42:20.962879 4004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d04ae63beb174bb49e79620d27c1d955" candidate_uuid: "e34072069e364e58b1ff988562f634c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9fed0f029de4108ba37caa508f5b9d0"
I20260807 19:42:20.963027 3859 raft_consensus.cc:3037] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:20.963054 4004 raft_consensus.cc:3037] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:20.965358 4004 raft_consensus.cc:2454] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e34072069e364e58b1ff988562f634c0 in term 1.
I20260807 19:42:20.965368 3859 raft_consensus.cc:2454] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e34072069e364e58b1ff988562f634c0 in term 1.
I20260807 19:42:20.965754 3645 leader_election.cc:302] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: c740b91a0f8f4f64b50853c4efaf86b9, e34072069e364e58b1ff988562f634c0, f9fed0f029de4108ba37caa508f5b9d0; no voters:
I20260807 19:42:20.966102 4081 raft_consensus.cc:2781] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:20.966549 4081 raft_consensus.cc:686] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [term 1 LEADER]: Becoming Leader. State: Replica: e34072069e364e58b1ff988562f634c0, State: Running, Role: LEADER
I20260807 19:42:20.967185 4081 consensus_queue.cc:237] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34072069e364e58b1ff988562f634c0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39771 } } peers { permanent_uuid: "c740b91a0f8f4f64b50853c4efaf86b9" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 35679 } } peers { permanent_uuid: "f9fed0f029de4108ba37caa508f5b9d0" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41281 } }
I20260807 19:42:20.971885 3559 catalog_manager.cc:5495] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e34072069e364e58b1ff988562f634c0 (127.30.208.1). New cstate: current_term: 1 leader_uuid: "e34072069e364e58b1ff988562f634c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34072069e364e58b1ff988562f634c0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c740b91a0f8f4f64b50853c4efaf86b9" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 35679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9fed0f029de4108ba37caa508f5b9d0" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41281 } health_report { overall_health: UNKNOWN } } }
I20260807 19:42:21.005533 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:20.999980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58162 (local address 127.30.208.1:39771)
0807 19:42:21.000237 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:21.000243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:21.000348 (+ 105us) server_negotiation.cc:407] Connection header received
0807 19:42:21.000590 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:21.000595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:21.000689 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:21.000824 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:21.001242 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:21.001977 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:21.003353 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:21.003761 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:21.003873 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:21.004379 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:21.004889 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:21.005024 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:21.005128 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 19:42:21.005266 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 19:42:21.013525 4004 raft_consensus.cc:1261] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0 [term 1 FOLLOWER]: Refusing update from remote peer e34072069e364e58b1ff988562f634c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:21.013525 3859 raft_consensus.cc:1261] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9 [term 1 FOLLOWER]: Refusing update from remote peer e34072069e364e58b1ff988562f634c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:21.014483 4081 consensus_queue.cc:1035] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9fed0f029de4108ba37caa508f5b9d0" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 41281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:21.014783 4084 consensus_queue.cc:1035] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c740b91a0f8f4f64b50853c4efaf86b9" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 35679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:21.029788 4087 mvcc.cc:205] Tried to move back new op lower bound from 7315995611181551616 to 7315995611005702144. Current Snapshot: MvccSnapshot[applied={T|T < 7315995611181551616}]
I20260807 19:42:21.030768 4089 mvcc.cc:205] Tried to move back new op lower bound from 7315995611181551616 to 7315995611005702144. Current Snapshot: MvccSnapshot[applied={T|T < 7315995611181551616}]
I20260807 19:42:21.049477 3559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:21.049780 3559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:21.053774 3559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ae3ea77d38d4635b610861f51bef397: Sending DeleteTablet for 3 replicas of tablet d04ae63beb174bb49e79620d27c1d955
I20260807 19:42:21.055089 3696 tablet_service.cc:1514] Processing DeleteTablet for tablet d04ae63beb174bb49e79620d27c1d955 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:21 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:58160
I20260807 19:42:21.055083 3838 tablet_service.cc:1514] Processing DeleteTablet for tablet d04ae63beb174bb49e79620d27c1d955 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:21 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:48150
I20260807 19:42:21.055164 3984 tablet_service.cc:1514] Processing DeleteTablet for tablet d04ae63beb174bb49e79620d27c1d955 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:21 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:49786
I20260807 19:42:21.057700 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 3625
I20260807 19:42:21.066582 4102 tablet_replica.cc:330] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0: stopping tablet replica
I20260807 19:42:21.067247 4102 raft_consensus.cc:2229] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:21.067955 4102 raft_consensus.cc:2258] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:21.070950 4102 ts_tablet_manager.cc:1905] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:21.071514 4101 tablet_replica.cc:330] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9: stopping tablet replica
I20260807 19:42:21.072468 4101 raft_consensus.cc:2229] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:21.075629 4101 raft_consensus.cc:2258] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:21.077805 4100 tablet_replica.cc:330] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0: stopping tablet replica
I20260807 19:42:21.078298 4100 raft_consensus.cc:2229] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:21.079031 4100 raft_consensus.cc:2258] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:21.079098 4102 ts_tablet_manager.cc:1918] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:21.079352 4102 log.cc:1196] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/d04ae63beb174bb49e79620d27c1d955
I20260807 19:42:21.079391 4101 ts_tablet_manager.cc:1905] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:21.080231 4102 ts_tablet_manager.cc:1939] T d04ae63beb174bb49e79620d27c1d955 P f9fed0f029de4108ba37caa508f5b9d0: Deleting consensus metadata
I20260807 19:42:21.081467 4100 ts_tablet_manager.cc:1905] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:21.082314 3545 catalog_manager.cc:4841] TS f9fed0f029de4108ba37caa508f5b9d0 (127.30.208.3:41281): tablet d04ae63beb174bb49e79620d27c1d955 (table test-table [id=871cfeafd9dd40f48c0ec191654b1e09]) successfully deleted
I20260807 19:42:21.086323 4101 ts_tablet_manager.cc:1918] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:21.086591 4101 log.cc:1196] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/d04ae63beb174bb49e79620d27c1d955
I20260807 19:42:21.087183 4101 ts_tablet_manager.cc:1939] T d04ae63beb174bb49e79620d27c1d955 P c740b91a0f8f4f64b50853c4efaf86b9: Deleting consensus metadata
Aug 07 19:42:21 dist-test-slave-vq2q krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:42:21.089251 3542 catalog_manager.cc:4841] TS c740b91a0f8f4f64b50853c4efaf86b9 (127.30.208.2:35679): tablet d04ae63beb174bb49e79620d27c1d955 (table test-table [id=871cfeafd9dd40f48c0ec191654b1e09]) successfully deleted
I20260807 19:42:21.089289 4100 ts_tablet_manager.cc:1918] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:21.089488 4100 log.cc:1196] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/d04ae63beb174bb49e79620d27c1d955
I20260807 19:42:21.090167 4100 ts_tablet_manager.cc:1939] T d04ae63beb174bb49e79620d27c1d955 P e34072069e364e58b1ff988562f634c0: Deleting consensus metadata
I20260807 19:42:21.091964 3545 catalog_manager.cc:4841] TS e34072069e364e58b1ff988562f634c0 (127.30.208.1:39771): tablet d04ae63beb174bb49e79620d27c1d955 (table test-table [id=871cfeafd9dd40f48c0ec191654b1e09]) successfully deleted
I20260807 19:42:21.095857 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:21.059036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58168 (local address 127.30.208.1:39771)
0807 19:42:21.059216 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:21.059222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:21.069135 (+ 9913us) server_negotiation.cc:407] Connection header received
0807 19:42:21.069427 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:21.069434 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:21.069532 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:21.069607 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:21.073083 (+ 3476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:21.074421 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:21.083007 (+ 8586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:21.083803 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:21.083931 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:21.089797 (+ 5866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:21.089810 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:21.089825 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:21.089877 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:21.092946 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:21.093708 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:21.093716 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:21.093718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:21.093874 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:21.094267 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:21.094271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:21.094273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:21.094654 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:21.094949 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:21.095282 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 19:42:21.095534 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 19:42:21.096606 3758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:21.213100 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 3625
I20260807 19:42:21.232028 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 3771
Aug 07 19:42:21 dist-test-slave-vq2q krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:42:21.246766 4069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:21.232858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48164 (local address 127.30.208.2:35679)
0807 19:42:21.233034 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:21.233040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:21.233059 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:42:21.233212 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:21.233218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:21.233290 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:21.233375 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:21.234604 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:21.235357 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:21.236558 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:21.236966 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:21.237080 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:21.241111 (+ 4031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:21.241128 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:21.241143 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:21.241251 (+ 108us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:21.244021 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:21.244692 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:21.244699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:21.244702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:21.244834 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:21.245211 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:21.245215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:21.245217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:21.245611 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:21.245919 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:21.246199 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 19:42:21.246415 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 19:42:21.247454 3900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:21.336143 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 3771
I20260807 19:42:21.353832 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 3911
Aug 07 19:42:21 dist-test-slave-vq2q krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:42:21.368709 4071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:21.354645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.30.208.3:41281)
0807 19:42:21.354832 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:21.354837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:21.354888 (+ 51us) server_negotiation.cc:407] Connection header received
0807 19:42:21.354991 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:21.354997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:21.355062 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:21.355147 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:21.356404 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:21.357110 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:21.358413 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:21.358829 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:21.358948 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:21.363190 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:21.363208 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:21.363224 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:21.363271 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:21.365967 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:21.366639 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:21.366647 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:21.366649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:21.366785 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:21.367176 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:21.367179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:21.367181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:21.367562 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:21.367845 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:21.368171 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 19:42:21.368394 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:42:21.369447 4045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:21.457828 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 3911
I20260807 19:42:21.475234 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 3527
I20260807 19:42:21.476456 3592 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:21.586009 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 3527
2026-08-07T19:42:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4313 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T19:42:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:21Z Disabled control of system clock
I20260807 19:42:21.655757 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:41301
--webserver_interface=127.30.208.62
--webserver_port=0
--builtin_ntp_servers=127.30.208.20:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:41301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 19:42:21.989603 4111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:21.989915 4111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:21.990005 4111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:21.997298 4111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:21.997423 4111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:21.997462 4111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:21.997520 4111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:21.997565 4111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:22.007592 4111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:41301
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:41301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:22.008917 4111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:22.010741 4111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:22.022742 4119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:22.022836 4117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:22.022857 4116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:22.023710 4111 server_base.cc:1034] running on GCE node
I20260807 19:42:22.024533 4111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:22.026149 4111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:22.027354 4111 hybrid_clock.cc:648] HybridClock initialized: now 1786131742027309 us; error 38 us; skew 500 ppm
I20260807 19:42:22.027696 4111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:22.030146 4111 webserver.cc:460] Webserver started at http://127.30.208.62:36785/ using document root <none> and password file <none>
I20260807 19:42:22.030947 4111 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:22.031062 4111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:22.031374 4111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:22.034490 4111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "07d34598e78e4a3eb7689cddc007875b"
format_stamp: "Formatted at 2026-08-07 19:42:22 on dist-test-slave-vq2q"
server_key: "514c525c32cf36fbffbcf2fb004a694b"
server_key_iv: "a2d623ecb05ad922ea027fc3082fb924"
server_key_version: "encryptionkey@0"
I20260807 19:42:22.035290 4111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "07d34598e78e4a3eb7689cddc007875b"
format_stamp: "Formatted at 2026-08-07 19:42:22 on dist-test-slave-vq2q"
server_key: "514c525c32cf36fbffbcf2fb004a694b"
server_key_iv: "a2d623ecb05ad922ea027fc3082fb924"
server_key_version: "encryptionkey@0"
I20260807 19:42:22.040549 4111 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:42:22.044219 4126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:22.045614 4111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:22.045785 4111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "07d34598e78e4a3eb7689cddc007875b"
format_stamp: "Formatted at 2026-08-07 19:42:22 on dist-test-slave-vq2q"
server_key: "514c525c32cf36fbffbcf2fb004a694b"
server_key_iv: "a2d623ecb05ad922ea027fc3082fb924"
server_key_version: "encryptionkey@0"
I20260807 19:42:22.045984 4111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:22.051606 4111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:22.052376 4111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:22.052628 4111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:22.069301 4111 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:41301
I20260807 19:42:22.069360 4183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:41301 every 8 connection(s)
I20260807 19:42:22.070669 4111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:22.074324 4184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:22.079819 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 4111
I20260807 19:42:22.080032 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:42:22.080646 31552 external_mini_cluster.cc:1428] Setting key 7b66787618e51cd1d596d8d12a604361
I20260807 19:42:22.085748 4184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b: Bootstrap starting.
I20260807 19:42:22.089097 4184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:22.089167 4187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:22.082745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42996 (local address 127.30.208.62:41301)
0807 19:42:22.083787 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:22.083798 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:22.083849 (+ 51us) server_negotiation.cc:407] Connection header received
0807 19:42:22.085565 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:22.085590 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:22.086071 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:22.086436 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:22.087554 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:22.087561 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:22.087574 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:22.087589 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:22.087660 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:22.087679 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:22.087890 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:22.088035 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 19:42:22.088279 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":335,"threads_started":1}
I20260807 19:42:22.090512 4184 log.cc:824] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:22.093743 4184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b: No bootstrap required, opened a new log
I20260807 19:42:22.101063 4184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07d34598e78e4a3eb7689cddc007875b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41301 } }
I20260807 19:42:22.101583 4184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:22.101698 4184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07d34598e78e4a3eb7689cddc007875b, State: Initialized, Role: FOLLOWER
I20260807 19:42:22.102309 4184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07d34598e78e4a3eb7689cddc007875b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41301 } }
I20260807 19:42:22.102540 4184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:22.102672 4184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:22.102831 4184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:22.105077 4184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07d34598e78e4a3eb7689cddc007875b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41301 } }
I20260807 19:42:22.105720 4184 leader_election.cc:302] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07d34598e78e4a3eb7689cddc007875b; no voters:
I20260807 19:42:22.106367 4184 leader_election.cc:288] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:22.106470 4189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:22.107177 4189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [term 1 LEADER]: Becoming Leader. State: Replica: 07d34598e78e4a3eb7689cddc007875b, State: Running, Role: LEADER
I20260807 19:42:22.107829 4189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07d34598e78e4a3eb7689cddc007875b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41301 } }
I20260807 19:42:22.108728 4184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:22.110590 4191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07d34598e78e4a3eb7689cddc007875b. Latest consensus state: current_term: 1 leader_uuid: "07d34598e78e4a3eb7689cddc007875b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07d34598e78e4a3eb7689cddc007875b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41301 } } }
I20260807 19:42:22.110999 4191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:22.113238 4190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07d34598e78e4a3eb7689cddc007875b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07d34598e78e4a3eb7689cddc007875b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41301 } } }
I20260807 19:42:22.113685 4190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:22.113876 4195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:22.120436 4195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:22.129959 4195 catalog_manager.cc:1347] Generated new cluster ID: 6e6696cc99474fadb3c5475d4777959a
I20260807 19:42:22.130091 4195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:22.156891 4195 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:22.157359 4195 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:22.164958 4195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b: Generated new TSK 0
I20260807 19:42:22.165836 4195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:22.175966 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41301
--builtin_ntp_servers=127.30.208.20:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 19:42:22.536762 4210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:22.537074 4210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:22.537168 4210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:22.544610 4210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:22.544715 4210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:22.544844 4210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:42:22.554409 4210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:22.555684 4210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:22.557562 4210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:22.570547 4220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:22.570524 4218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:22.571098 4217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:22.571571 4210 server_base.cc:1034] running on GCE node
I20260807 19:42:22.572160 4210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:22.573099 4210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:22.574323 4210 hybrid_clock.cc:648] HybridClock initialized: now 1786131742574279 us; error 34 us; skew 500 ppm
I20260807 19:42:22.574628 4210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:22.576836 4210 webserver.cc:460] Webserver started at http://127.30.208.1:37655/ using document root <none> and password file <none>
I20260807 19:42:22.577577 4210 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:22.577679 4210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:22.577999 4210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:22.580799 4210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "683ea6e3241b4e949ba5de1bc876af06"
format_stamp: "Formatted at 2026-08-07 19:42:22 on dist-test-slave-vq2q"
server_key: "ab74514464a4a746324847410959ee53"
server_key_iv: "730b2c3746e8fdc93c1db041e6aa2114"
server_key_version: "encryptionkey@0"
I20260807 19:42:22.581588 4210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "683ea6e3241b4e949ba5de1bc876af06"
format_stamp: "Formatted at 2026-08-07 19:42:22 on dist-test-slave-vq2q"
server_key: "ab74514464a4a746324847410959ee53"
server_key_iv: "730b2c3746e8fdc93c1db041e6aa2114"
server_key_version: "encryptionkey@0"
I20260807 19:42:22.586583 4210 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:42:22.590186 4226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:22.591549 4210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 19:42:22.591715 4210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "683ea6e3241b4e949ba5de1bc876af06"
format_stamp: "Formatted at 2026-08-07 19:42:22 on dist-test-slave-vq2q"
server_key: "ab74514464a4a746324847410959ee53"
server_key_iv: "730b2c3746e8fdc93c1db041e6aa2114"
server_key_version: "encryptionkey@0"
I20260807 19:42:22.591950 4210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:22.597460 4210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:22.598253 4210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:22.598501 4210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:22.599390 4210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:22.600862 4210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:22.600960 4210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:22.601068 4210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:22.601137 4210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:22.631351 4210 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:44845
I20260807 19:42:22.631404 4349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:44845 every 8 connection(s)
I20260807 19:42:22.632643 4210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:22.640561 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:22.635107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:54471 (local address 127.30.208.62:41301)
0807 19:42:22.635508 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:22.635513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:22.636606 (+ 1093us) server_negotiation.cc:407] Connection header received
0807 19:42:22.638349 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:22.638355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:22.638424 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:22.638524 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:22.639796 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:22.639801 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:22.639805 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:22.639821 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:22.639875 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:22.639888 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:22.639997 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:22.640188 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 19:42:22.640258 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":213,"threads_started":1}
I20260807 19:42:22.641535 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:22.635576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:41301 (local address 127.30.208.1:54471)
0807 19:42:22.636445 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:22.636484 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:22.638066 (+ 1582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:22.638760 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:22.638769 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:22.638860 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:22.639486 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:22.639489 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:22.639527 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:22.639614 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:22.639985 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:22.639995 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:22.640003 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 19:42:22.640257 (+ 254us) client_negotiation.cc:240] Negotiation successful
0807 19:42:22.640546 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":254,"threads_started":1}
I20260807 19:42:22.642370 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 4210
I20260807 19:42:22.642606 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:22.643116 4350 heartbeater.cc:347] Connected to a master server at 127.30.208.62:41301
I20260807 19:42:22.643380 31552 external_mini_cluster.cc:1428] Setting key 815e7b6e4e8e8d6c18626d6b2373c479
I20260807 19:42:22.643414 4350 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:22.644006 4350 heartbeater.cc:511] Master 127.30.208.62:41301 requested a full tablet report, sending...
I20260807 19:42:22.646564 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41301
--builtin_ntp_servers=127.30.208.20:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 19:42:22.646593 4145 ts_manager.cc:194] Registered new tserver with Master: 683ea6e3241b4e949ba5de1bc876af06 (127.30.208.1:44845)
W20260807 19:42:22.978533 4354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:22.978832 4354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:22.978951 4354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:22.986392 4354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:22.986511 4354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:22.986657 4354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:42:22.995977 4354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:22.997298 4354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:22.999069 4354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:23.011682 4359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:23.011755 4360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:23.011721 4362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:23.012705 4354 server_base.cc:1034] running on GCE node
I20260807 19:42:23.013418 4354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:23.014369 4354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:23.015600 4354 hybrid_clock.cc:648] HybridClock initialized: now 1786131743015541 us; error 49 us; skew 500 ppm
I20260807 19:42:23.015952 4354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:23.018467 4354 webserver.cc:460] Webserver started at http://127.30.208.2:45789/ using document root <none> and password file <none>
I20260807 19:42:23.019166 4354 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:23.019271 4354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:23.019593 4354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:23.022625 4354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "f28d51c777904fea93721be6b2899974"
format_stamp: "Formatted at 2026-08-07 19:42:23 on dist-test-slave-vq2q"
server_key: "279dec058d63baea39628b2cf37bf4a5"
server_key_iv: "03f2fbb2af6086901866055bf46717d3"
server_key_version: "encryptionkey@0"
I20260807 19:42:23.023569 4354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "f28d51c777904fea93721be6b2899974"
format_stamp: "Formatted at 2026-08-07 19:42:23 on dist-test-slave-vq2q"
server_key: "279dec058d63baea39628b2cf37bf4a5"
server_key_iv: "03f2fbb2af6086901866055bf46717d3"
server_key_version: "encryptionkey@0"
I20260807 19:42:23.028952 4354 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 19:42:23.032332 4370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:23.033638 4354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 19:42:23.033811 4354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "f28d51c777904fea93721be6b2899974"
format_stamp: "Formatted at 2026-08-07 19:42:23 on dist-test-slave-vq2q"
server_key: "279dec058d63baea39628b2cf37bf4a5"
server_key_iv: "03f2fbb2af6086901866055bf46717d3"
server_key_version: "encryptionkey@0"
I20260807 19:42:23.034014 4354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:23.039569 4354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:23.040337 4354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:23.040584 4354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:23.041518 4354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:23.042930 4354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:23.043027 4354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:23.043136 4354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:23.043206 4354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:23.074041 4354 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:45907
I20260807 19:42:23.074090 4489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:45907 every 8 connection(s)
I20260807 19:42:23.075672 4354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:23.079017 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 4354
I20260807 19:42:23.079200 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:23.079720 31552 external_mini_cluster.cc:1428] Setting key 0db7c62fa74990c01348a106d951de8f
I20260807 19:42:23.083076 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41301
--builtin_ntp_servers=127.30.208.20:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 19:42:23.084704 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.078555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:41357 (local address 127.30.208.62:41301)
0807 19:42:23.078754 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.078761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.080427 (+ 1666us) server_negotiation.cc:407] Connection header received
0807 19:42:23.082540 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.082546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.082610 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.082694 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.083968 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.083974 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.083979 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.083999 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.084036 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.084047 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.084159 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.084339 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.084406 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:42:23.085836 4492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.079119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:41301 (local address 127.30.208.2:41357)
0807 19:42:23.080258 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.080307 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.082268 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.082936 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.082944 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.083034 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:23.083676 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:23.083679 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:23.083717 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:23.083802 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:23.084134 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:23.084146 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:23.084154 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.084435 (+ 281us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.084699 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":326,"threads_started":1}
I20260807 19:42:23.087159 4490 heartbeater.cc:347] Connected to a master server at 127.30.208.62:41301
I20260807 19:42:23.087451 4490 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:23.088023 4490 heartbeater.cc:511] Master 127.30.208.62:41301 requested a full tablet report, sending...
I20260807 19:42:23.089501 4145 ts_manager.cc:194] Registered new tserver with Master: f28d51c777904fea93721be6b2899974 (127.30.208.2:45907)
W20260807 19:42:23.420470 4493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:23.420755 4493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:23.420881 4493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:23.428401 4493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:23.428503 4493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:23.428637 4493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:42:23.438040 4493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:23.439354 4493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:23.441114 4493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:23.453776 4498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:23.453925 4501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:23.453923 4499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:23.454694 4493 server_base.cc:1034] running on GCE node
I20260807 19:42:23.455261 4493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:23.456173 4493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:23.457392 4493 hybrid_clock.cc:648] HybridClock initialized: now 1786131743457335 us; error 49 us; skew 500 ppm
I20260807 19:42:23.457672 4493 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:23.459743 4493 webserver.cc:460] Webserver started at http://127.30.208.3:36855/ using document root <none> and password file <none>
I20260807 19:42:23.460486 4493 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:23.460582 4493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:23.460918 4493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:23.463798 4493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "86357eb77786454ea82f990f0e22089e"
format_stamp: "Formatted at 2026-08-07 19:42:23 on dist-test-slave-vq2q"
server_key: "9809c68b5b27cca7f4eeed7b8240c815"
server_key_iv: "d5c953ff02df283be864ac1e9cab3fc2"
server_key_version: "encryptionkey@0"
I20260807 19:42:23.464517 4493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "86357eb77786454ea82f990f0e22089e"
format_stamp: "Formatted at 2026-08-07 19:42:23 on dist-test-slave-vq2q"
server_key: "9809c68b5b27cca7f4eeed7b8240c815"
server_key_iv: "d5c953ff02df283be864ac1e9cab3fc2"
server_key_version: "encryptionkey@0"
I20260807 19:42:23.469481 4493 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 19:42:23.472745 4508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:23.474048 4493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:23.474172 4493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "86357eb77786454ea82f990f0e22089e"
format_stamp: "Formatted at 2026-08-07 19:42:23 on dist-test-slave-vq2q"
server_key: "9809c68b5b27cca7f4eeed7b8240c815"
server_key_iv: "d5c953ff02df283be864ac1e9cab3fc2"
server_key_version: "encryptionkey@0"
I20260807 19:42:23.474354 4493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:23.479699 4493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:23.480414 4493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:23.480646 4493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:23.481544 4493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:23.482890 4493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:23.482978 4493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:23.483064 4493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:23.483117 4493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:23.512487 4493 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:38147
I20260807 19:42:23.512542 4633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:38147 every 8 connection(s)
I20260807 19:42:23.513813 4493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:23.515837 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 4493
I20260807 19:42:23.516034 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:23.516625 31552 external_mini_cluster.cc:1428] Setting key b223eca1710de68ddec4c751a86ae23f
I20260807 19:42:23.521976 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.516373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:42575 (local address 127.30.208.62:41301)
0807 19:42:23.516544 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.516548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.518312 (+ 1764us) server_negotiation.cc:407] Connection header received
0807 19:42:23.519925 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.519930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.519985 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.520063 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.521362 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.521367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.521370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.521388 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.521420 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.521433 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.521540 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.521690 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.521768 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:42:23.522908 4636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.517083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:41301 (local address 127.30.208.3:42575)
0807 19:42:23.518147 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.518202 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.519674 (+ 1472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.520254 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.520262 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.520351 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:23.521012 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:23.521014 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:23.521052 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:23.521140 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:23.521525 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:23.521532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:23.521538 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.521764 (+ 226us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.521976 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":311,"threads_started":1}
I20260807 19:42:23.524135 4634 heartbeater.cc:347] Connected to a master server at 127.30.208.62:41301
I20260807 19:42:23.524415 4634 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:23.524994 4634 heartbeater.cc:511] Master 127.30.208.62:41301 requested a full tablet report, sending...
I20260807 19:42:23.526461 4145 ts_manager.cc:194] Registered new tserver with Master: 86357eb77786454ea82f990f0e22089e (127.30.208.3:38147)
I20260807 19:42:23.531891 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:42:23.538484 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.536106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43000 (local address 127.30.208.62:41301)
0807 19:42:23.536252 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.536257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.536763 (+ 506us) server_negotiation.cc:407] Connection header received
0807 19:42:23.536915 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.536920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.536984 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.537069 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.537915 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.537919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.537922 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.537937 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.537970 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.537983 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.538096 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.538157 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.538205 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 19:42:23.542214 4145 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:23.549093 4145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:23.568974 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.562037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:45907 (local address 127.0.0.1:35358)
0807 19:42:23.563982 (+ 1945us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.564001 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.564096 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.566330 (+ 2234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.566334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.566359 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:23.567113 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:23.567116 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:23.567133 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:23.567186 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:23.568172 (+ 986us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:23.568177 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:23.568181 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.568347 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.568436 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1803,"mutex_wait_us":26,"thread_start_us":2236,"threads_started":1}
I20260807 19:42:23.569098 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.563052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54504 (local address 127.30.208.1:44845)
0807 19:42:23.565370 (+ 2318us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.565378 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.565406 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:42:23.565479 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.565487 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.565748 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.566517 (+ 769us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.567869 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.567877 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.567891 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.567908 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.567964 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.567990 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.568655 (+ 665us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.568723 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.568824 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2140,"thread_start_us":2193,"threads_started":1}
I20260807 19:42:23.569216 4647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.561303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:38147 (local address 127.0.0.1:59976)
0807 19:42:23.562728 (+ 1425us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.562771 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.562909 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.566781 (+ 3872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.566785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.566807 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:23.567435 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:23.567438 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:23.567454 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:23.567494 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:23.568478 (+ 984us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:23.568483 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:23.568488 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.568625 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.568686 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1127,"thread_start_us":1731,"threads_started":1}
I20260807 19:42:23.569700 4648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.561482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59976 (local address 127.30.208.3:38147)
0807 19:42:23.566233 (+ 4751us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.566240 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.566279 (+ 39us) server_negotiation.cc:407] Connection header received
0807 19:42:23.566344 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.566351 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.566561 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.567213 (+ 652us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.568236 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.568244 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.568256 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.568275 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.568329 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.568350 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.569132 (+ 782us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.569245 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.569335 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4608,"thread_start_us":4438,"threads_started":1}
I20260807 19:42:23.569803 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.562492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.0.0.1:54504)
0807 19:42:23.564515 (+ 2023us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.564531 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.564610 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.565967 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.565975 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.566061 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:23.566821 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:23.566824 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:23.566871 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:23.566962 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:23.568119 (+ 1157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:23.568131 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:23.568139 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.568308 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.568424 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1921,"thread_start_us":1226,"threads_started":1}
I20260807 19:42:23.570175 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.561732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35358 (local address 127.30.208.2:45907)
0807 19:42:23.565851 (+ 4119us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.565859 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.565885 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:42:23.565951 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.565958 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.566149 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.566303 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.567956 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.567963 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.567973 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.567996 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.568049 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.568069 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.569652 (+ 1583us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.569744 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.569835 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3937,"thread_start_us":3793,"threads_started":1}
I20260807 19:42:23.573283 4563 tablet_service.cc:1467] Processing CreateTablet for tablet aa2c1e7f666949d88f4ef6695d88bffa (DEFAULT_TABLE table=test-table [id=84e0cd5682244b1da8f51fc42c6eaed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:23.573293 4280 tablet_service.cc:1467] Processing CreateTablet for tablet aa2c1e7f666949d88f4ef6695d88bffa (DEFAULT_TABLE table=test-table [id=84e0cd5682244b1da8f51fc42c6eaed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:23.573293 4421 tablet_service.cc:1467] Processing CreateTablet for tablet aa2c1e7f666949d88f4ef6695d88bffa (DEFAULT_TABLE table=test-table [id=84e0cd5682244b1da8f51fc42c6eaed6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:23.574483 4563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa2c1e7f666949d88f4ef6695d88bffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:23.574558 4421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa2c1e7f666949d88f4ef6695d88bffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:23.574554 4280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa2c1e7f666949d88f4ef6695d88bffa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:23.586165 4653 tablet_bootstrap.cc:492] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e: Bootstrap starting.
I20260807 19:42:23.588490 4655 tablet_bootstrap.cc:492] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974: Bootstrap starting.
I20260807 19:42:23.589999 4654 tablet_bootstrap.cc:492] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06: Bootstrap starting.
I20260807 19:42:23.590829 4653 tablet_bootstrap.cc:654] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:23.591678 4655 tablet_bootstrap.cc:654] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:23.592173 4653 log.cc:824] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:23.592861 4655 log.cc:824] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:23.592994 4654 tablet_bootstrap.cc:654] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:23.594071 4654 log.cc:824] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:23.595453 4653 tablet_bootstrap.cc:492] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e: No bootstrap required, opened a new log
I20260807 19:42:23.595671 4655 tablet_bootstrap.cc:492] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974: No bootstrap required, opened a new log
I20260807 19:42:23.595734 4653 ts_tablet_manager.cc:1397] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 19:42:23.595940 4655 ts_tablet_manager.cc:1397] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 19:42:23.597071 4654 tablet_bootstrap.cc:492] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06: No bootstrap required, opened a new log
I20260807 19:42:23.597370 4654 ts_tablet_manager.cc:1397] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260807 19:42:23.603276 4653 raft_consensus.cc:348] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86357eb77786454ea82f990f0e22089e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38147 } } peers { permanent_uuid: "f28d51c777904fea93721be6b2899974" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45907 } } peers { permanent_uuid: "683ea6e3241b4e949ba5de1bc876af06" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44845 } }
I20260807 19:42:23.603658 4653 raft_consensus.cc:374] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:23.603778 4653 raft_consensus.cc:729] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86357eb77786454ea82f990f0e22089e, State: Initialized, Role: FOLLOWER
I20260807 19:42:23.604378 4653 consensus_queue.cc:260] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86357eb77786454ea82f990f0e22089e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38147 } } peers { permanent_uuid: "f28d51c777904fea93721be6b2899974" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45907 } } peers { permanent_uuid: "683ea6e3241b4e949ba5de1bc876af06" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44845 } }
I20260807 19:42:23.605536 4634 heartbeater.cc:503] Master 127.30.208.62:41301 was elected leader, sending a full tablet report...
I20260807 19:42:23.606247 4653 ts_tablet_manager.cc:1428] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260807 19:42:23.606232 4655 raft_consensus.cc:348] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86357eb77786454ea82f990f0e22089e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38147 } } peers { permanent_uuid: "f28d51c777904fea93721be6b2899974" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45907 } } peers { permanent_uuid: "683ea6e3241b4e949ba5de1bc876af06" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44845 } }
I20260807 19:42:23.606680 4655 raft_consensus.cc:374] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:23.606765 4655 raft_consensus.cc:729] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f28d51c777904fea93721be6b2899974, State: Initialized, Role: FOLLOWER
I20260807 19:42:23.607370 4654 raft_consensus.cc:348] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86357eb77786454ea82f990f0e22089e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38147 } } peers { permanent_uuid: "f28d51c777904fea93721be6b2899974" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45907 } } peers { permanent_uuid: "683ea6e3241b4e949ba5de1bc876af06" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44845 } }
I20260807 19:42:23.607501 4655 consensus_queue.cc:260] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86357eb77786454ea82f990f0e22089e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38147 } } peers { permanent_uuid: "f28d51c777904fea93721be6b2899974" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45907 } } peers { permanent_uuid: "683ea6e3241b4e949ba5de1bc876af06" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44845 } }
I20260807 19:42:23.607795 4654 raft_consensus.cc:374] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:23.607915 4654 raft_consensus.cc:729] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 683ea6e3241b4e949ba5de1bc876af06, State: Initialized, Role: FOLLOWER
I20260807 19:42:23.608620 4490 heartbeater.cc:503] Master 127.30.208.62:41301 was elected leader, sending a full tablet report...
I20260807 19:42:23.608803 4654 consensus_queue.cc:260] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86357eb77786454ea82f990f0e22089e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38147 } } peers { permanent_uuid: "f28d51c777904fea93721be6b2899974" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45907 } } peers { permanent_uuid: "683ea6e3241b4e949ba5de1bc876af06" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44845 } }
I20260807 19:42:23.609323 4655 ts_tablet_manager.cc:1428] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260807 19:42:23.611210 4350 heartbeater.cc:503] Master 127.30.208.62:41301 was elected leader, sending a full tablet report...
I20260807 19:42:23.613215 4654 ts_tablet_manager.cc:1428] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260807 19:42:23.614718 4661 raft_consensus.cc:482] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:23.614954 4661 raft_consensus.cc:504] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86357eb77786454ea82f990f0e22089e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38147 } } peers { permanent_uuid: "f28d51c777904fea93721be6b2899974" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45907 } } peers { permanent_uuid: "683ea6e3241b4e949ba5de1bc876af06" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44845 } }
I20260807 19:42:23.616442 4661 leader_election.cc:288] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86357eb77786454ea82f990f0e22089e (127.30.208.3:38147), 683ea6e3241b4e949ba5de1bc876af06 (127.30.208.1:44845)
I20260807 19:42:23.619714 4648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.617052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:51769 (local address 127.30.208.3:38147)
0807 19:42:23.617264 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.617271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.617765 (+ 494us) server_negotiation.cc:407] Connection header received
0807 19:42:23.617933 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.617937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.617988 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.618073 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.619153 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.619158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.619162 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.619182 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.619221 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.619236 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.619358 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.619409 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.619464 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:42:23.619716 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.617271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:36021 (local address 127.30.208.1:44845)
0807 19:42:23.617438 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.617442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.617761 (+ 319us) server_negotiation.cc:407] Connection header received
0807 19:42:23.617915 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.617919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.617976 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.618076 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.619169 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.619175 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.619178 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.619195 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.619225 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.619239 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.619366 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.619409 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.619464 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:42:23.620323 4663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.616644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:38147 (local address 127.30.208.2:51769)
0807 19:42:23.617670 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.617684 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.617839 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.618111 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.618116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.618142 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:23.618973 (+ 831us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:23.618976 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:23.618987 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:23.619014 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:23.619323 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:23.619327 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:23.619331 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.619478 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.619543 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"spinlock_wait_cycles":32512,"thread_start_us":347,"threads_started":1}
I20260807 19:42:23.620422 4664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.616821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.30.208.2:36021)
0807 19:42:23.617658 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.617678 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:23.617800 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:23.618111 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:23.618116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:23.618142 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:23.618939 (+ 797us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:42:23.618943 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:23.618961 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:42:23.619011 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:23.619319 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:23.619324 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:23.619328 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 19:42:23.619486 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 19:42:23.619557 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":22,"thread_start_us":497,"threads_started":1}
I20260807 19:42:23.621281 4301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa2c1e7f666949d88f4ef6695d88bffa" candidate_uuid: "f28d51c777904fea93721be6b2899974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "683ea6e3241b4e949ba5de1bc876af06" is_pre_election: true
I20260807 19:42:23.621281 4583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa2c1e7f666949d88f4ef6695d88bffa" candidate_uuid: "f28d51c777904fea93721be6b2899974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86357eb77786454ea82f990f0e22089e" is_pre_election: true
I20260807 19:42:23.621819 4583 raft_consensus.cc:2454] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f28d51c777904fea93721be6b2899974 in term 0.
I20260807 19:42:23.621848 4301 raft_consensus.cc:2454] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f28d51c777904fea93721be6b2899974 in term 0.
I20260807 19:42:23.622509 4372 leader_election.cc:302] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 683ea6e3241b4e949ba5de1bc876af06, f28d51c777904fea93721be6b2899974; no voters:
I20260807 19:42:23.622912 4661 raft_consensus.cc:2781] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:23.623055 4661 raft_consensus.cc:482] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:23.623152 4661 raft_consensus.cc:3037] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:23.625785 4661 raft_consensus.cc:504] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86357eb77786454ea82f990f0e22089e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38147 } } peers { permanent_uuid: "f28d51c777904fea93721be6b2899974" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45907 } } peers { permanent_uuid: "683ea6e3241b4e949ba5de1bc876af06" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44845 } }
I20260807 19:42:23.626327 4661 leader_election.cc:288] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [CANDIDATE]: Term 1 election: Requested vote from peers 86357eb77786454ea82f990f0e22089e (127.30.208.3:38147), 683ea6e3241b4e949ba5de1bc876af06 (127.30.208.1:44845)
I20260807 19:42:23.626741 4583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa2c1e7f666949d88f4ef6695d88bffa" candidate_uuid: "f28d51c777904fea93721be6b2899974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86357eb77786454ea82f990f0e22089e"
I20260807 19:42:23.626746 4301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa2c1e7f666949d88f4ef6695d88bffa" candidate_uuid: "f28d51c777904fea93721be6b2899974" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "683ea6e3241b4e949ba5de1bc876af06"
I20260807 19:42:23.626922 4301 raft_consensus.cc:3037] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:23.626933 4583 raft_consensus.cc:3037] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:23.629077 4583 raft_consensus.cc:2454] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f28d51c777904fea93721be6b2899974 in term 1.
I20260807 19:42:23.629133 4301 raft_consensus.cc:2454] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f28d51c777904fea93721be6b2899974 in term 1.
I20260807 19:42:23.629487 4374 leader_election.cc:302] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86357eb77786454ea82f990f0e22089e, f28d51c777904fea93721be6b2899974; no voters:
I20260807 19:42:23.629786 4661 raft_consensus.cc:2781] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:23.630191 4661 raft_consensus.cc:686] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [term 1 LEADER]: Becoming Leader. State: Replica: f28d51c777904fea93721be6b2899974, State: Running, Role: LEADER
I20260807 19:42:23.630692 4661 consensus_queue.cc:237] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86357eb77786454ea82f990f0e22089e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38147 } } peers { permanent_uuid: "f28d51c777904fea93721be6b2899974" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45907 } } peers { permanent_uuid: "683ea6e3241b4e949ba5de1bc876af06" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44845 } }
W20260807 19:42:23.634904 4351 tablet.cc:2369] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:23.635634 4143 catalog_manager.cc:5495] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 reported cstate change: term changed from 0 to 1, leader changed from <none> to f28d51c777904fea93721be6b2899974 (127.30.208.2). New cstate: current_term: 1 leader_uuid: "f28d51c777904fea93721be6b2899974" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86357eb77786454ea82f990f0e22089e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f28d51c777904fea93721be6b2899974" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "683ea6e3241b4e949ba5de1bc876af06" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44845 } health_report { overall_health: UNKNOWN } } }
I20260807 19:42:23.648483 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.646696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35370 (local address 127.30.208.2:45907)
0807 19:42:23.646860 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.646864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.646882 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:42:23.647019 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.647023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.647099 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.647178 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.647931 (+ 753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.647938 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.647942 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.647960 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.647997 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.648013 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.648141 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.648186 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.648240 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:42:23.656292 4301 raft_consensus.cc:1261] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06 [term 1 FOLLOWER]: Refusing update from remote peer f28d51c777904fea93721be6b2899974: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:23.656287 4583 raft_consensus.cc:1261] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e [term 1 FOLLOWER]: Refusing update from remote peer f28d51c777904fea93721be6b2899974: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:23.657294 4665 consensus_queue.cc:1035] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "683ea6e3241b4e949ba5de1bc876af06" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:23.657634 4661 consensus_queue.cc:1035] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86357eb77786454ea82f990f0e22089e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:23.668454 4669 mvcc.cc:205] Tried to move back new op lower bound from 7315995622006788096 to 7315995621915369472. Current Snapshot: MvccSnapshot[applied={T|T < 7315995622006788096}]
I20260807 19:42:23.683651 4143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:23.683923 4143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:23.688627 4143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 07d34598e78e4a3eb7689cddc007875b: Sending DeleteTablet for 3 replicas of tablet aa2c1e7f666949d88f4ef6695d88bffa
I20260807 19:42:23.689811 4421 tablet_service.cc:1514] Processing DeleteTablet for tablet aa2c1e7f666949d88f4ef6695d88bffa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:23 UTC) from {username='slave'} at 127.0.0.1:35358
I20260807 19:42:23.689842 4563 tablet_service.cc:1514] Processing DeleteTablet for tablet aa2c1e7f666949d88f4ef6695d88bffa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:23 UTC) from {username='slave'} at 127.0.0.1:59976
I20260807 19:42:23.690069 4280 tablet_service.cc:1514] Processing DeleteTablet for tablet aa2c1e7f666949d88f4ef6695d88bffa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:23 UTC) from {username='slave'} at 127.0.0.1:54504
I20260807 19:42:23.690500 4279 tablet_service.cc:1514] Processing DeleteTablet for tablet aa2c1e7f666949d88f4ef6695d88bffa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:23 UTC) from {username='slave'} at 127.0.0.1:54504
I20260807 19:42:23.691308 4678 tablet_replica.cc:330] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974: stopping tablet replica
I20260807 19:42:23.691428 4679 tablet_replica.cc:330] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e: stopping tablet replica
I20260807 19:42:23.691767 4678 raft_consensus.cc:2229] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:23.691905 4679 raft_consensus.cc:2229] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:23.692299 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 4210
I20260807 19:42:23.692454 4679 raft_consensus.cc:2258] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:23.692446 4678 raft_consensus.cc:2258] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:23.695515 4679 ts_tablet_manager.cc:1905] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:23.698372 4678 ts_tablet_manager.cc:1905] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:23.701435 4680 tablet_replica.cc:330] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06: stopping tablet replica
I20260807 19:42:23.701880 4680 raft_consensus.cc:2229] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:23.702406 4680 raft_consensus.cc:2258] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:23.702648 4679 ts_tablet_manager.cc:1918] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:23.703089 4679 log.cc:1196] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/aa2c1e7f666949d88f4ef6695d88bffa
I20260807 19:42:23.703714 4679 ts_tablet_manager.cc:1939] T aa2c1e7f666949d88f4ef6695d88bffa P 86357eb77786454ea82f990f0e22089e: Deleting consensus metadata
I20260807 19:42:23.705861 4130 catalog_manager.cc:4841] TS 86357eb77786454ea82f990f0e22089e (127.30.208.3:38147): tablet aa2c1e7f666949d88f4ef6695d88bffa (table test-table [id=84e0cd5682244b1da8f51fc42c6eaed6]) successfully deleted
I20260807 19:42:23.708359 4680 ts_tablet_manager.cc:1905] T aa2c1e7f666949d88f4ef6695d88bffa P 683ea6e3241b4e949ba5de1bc876af06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:23.709506 4678 ts_tablet_manager.cc:1918] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:23.710565 4678 log.cc:1196] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/aa2c1e7f666949d88f4ef6695d88bffa
I20260807 19:42:23.711278 4678 ts_tablet_manager.cc:1939] T aa2c1e7f666949d88f4ef6695d88bffa P f28d51c777904fea93721be6b2899974: Deleting consensus metadata
I20260807 19:42:23.712746 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.693750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54508 (local address 127.30.208.1:44845)
0807 19:42:23.693927 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.693933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.710242 (+ 16309us) server_negotiation.cc:407] Connection header received
0807 19:42:23.710434 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.710443 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.710552 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.710662 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.711934 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.711940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.711945 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.711965 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.712003 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.712019 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.712122 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.712376 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.712447 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:42:23.713553 4129 catalog_manager.cc:4841] TS f28d51c777904fea93721be6b2899974 (127.30.208.2:45907): tablet aa2c1e7f666949d88f4ef6695d88bffa (table test-table [id=84e0cd5682244b1da8f51fc42c6eaed6]) successfully deleted
I20260807 19:42:23.713893 4345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:23.804759 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 4210
W20260807 19:42:23.829905 4128 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.208.1:44845 (error 108)
I20260807 19:42:23.830132 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 4354
I20260807 19:42:23.832028 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.831598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.0.0.1:54524)
0807 19:42:23.831771 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.831896 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 19:42:23.832327 4128 catalog_manager.cc:4584] TS 683ea6e3241b4e949ba5de1bc876af06 (127.30.208.1:44845): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aa2c1e7f666949d88f4ef6695d88bffa: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44845: connect: Connection refused (error 111)
I20260807 19:42:23.833746 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.830953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35378 (local address 127.30.208.2:45907)
0807 19:42:23.831170 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.831177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.831599 (+ 422us) server_negotiation.cc:407] Connection header received
0807 19:42:23.831831 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.831837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.831909 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.831991 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.833093 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.833098 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.833101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.833116 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.833150 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.833162 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.833332 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.833429 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.833475 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:42:23.834540 4485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:23.877471 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.876845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.0.0.1:54540)
0807 19:42:23.877107 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.877257 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 19:42:23.878571 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.878180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.0.0.1:54550)
0807 19:42:23.878341 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.878433 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 19:42:23.885763 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.885282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.0.0.1:54556)
0807 19:42:23.885457 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.885585 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 19:42:23.886729 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.886380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.0.0.1:54564)
0807 19:42:23.886520 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.886601 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 19:42:23.927361 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.926789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.0.0.1:54566)
0807 19:42:23.927089 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.927190 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 19:42:23.928598 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.928146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.0.0.1:54580)
0807 19:42:23.928393 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.928470 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260807 19:42:23.945349 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 4354
I20260807 19:42:23.963330 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.962856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.0.0.1:54584)
0807 19:42:23.963064 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.963153 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 19:42:23.964350 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.963984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.0.0.1:54600)
0807 19:42:23.964128 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:23.964211 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 19:42:23.964789 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 4493
I20260807 19:42:23.967558 4648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:23.965571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59986 (local address 127.30.208.3:38147)
0807 19:42:23.965846 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:23.965852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:23.965871 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:42:23.965924 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:23.965928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:23.966015 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:23.966100 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:23.966944 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:23.966948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:23.966952 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:23.966968 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:23.967000 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:23.967012 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:23.967123 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:23.967221 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 19:42:23.967283 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:42:23.968261 4628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:24.029259 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:24.028635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.0.0.1:54606)
0807 19:42:24.028898 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:24.029011 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 19:42:24.030386 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:24.029995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.0.0.1:54610)
0807 19:42:24.030147 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:24.030244 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 19:42:24.056777 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 4493
I20260807 19:42:24.065888 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:24.065398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.0.0.1:54614)
0807 19:42:24.065598 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:24.065714 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 19:42:24.066845 4650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:24.066507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44845 (local address 127.0.0.1:54628)
0807 19:42:24.066650 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:24.066720 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 19:42:24.073827 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 4111
I20260807 19:42:24.075013 4178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:24.183930 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 4111
2026-08-07T19:42:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2596 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 19:42:24.225203 31552 mini_oidc.cc:152] Starting JWKS server
I20260807 19:42:24.228245 31552 webserver.cc:460] Webserver started at https://127.0.0.1:43505/ using document root <none> and password file <none>
I20260807 19:42:24.228605 31552 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 19:42:24.229367 31552 webserver.cc:460] Webserver started at http://127.0.0.1:46361/ using document root <none> and password file <none>
2026-08-07T19:42:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:24Z Disabled control of system clock
I20260807 19:42:24.261235 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:37907
--webserver_interface=127.30.208.62
--webserver_port=0
--builtin_ntp_servers=127.30.208.20:37925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:37907
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43505/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/testchainca.cert with env {}
W20260807 19:42:24.586884 4694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:24.587181 4694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:24.587270 4694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:24.594426 4694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:24.594539 4694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:24.594590 4694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:24.594650 4694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:24.594700 4694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:42:24.594762 4694 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:42:24.594801 4694 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43505/jwks.json
I20260807 19:42:24.604635 4694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:37907
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:37907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43505/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:24.605908 4694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:24.607158 4694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:24.618719 4699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:24.618796 4700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:24.618758 4703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:24.619717 4694 server_base.cc:1034] running on GCE node
I20260807 19:42:24.620540 4694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:24.622040 4694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:24.623252 4694 hybrid_clock.cc:648] HybridClock initialized: now 1786131744623204 us; error 39 us; skew 500 ppm
I20260807 19:42:24.623629 4694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:24.626030 4694 webserver.cc:460] Webserver started at http://127.30.208.62:39739/ using document root <none> and password file <none>
I20260807 19:42:24.626734 4694 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:24.626835 4694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:24.627148 4694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:24.630296 4694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "545178215a9a47a7a8a76d2e36349ee5"
format_stamp: "Formatted at 2026-08-07 19:42:24 on dist-test-slave-vq2q"
server_key: "95d5209ab47a40913db3cb176df6bb70"
server_key_iv: "cdc3919304d4669c1674aa843d1feaa2"
server_key_version: "encryptionkey@0"
I20260807 19:42:24.631202 4694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "545178215a9a47a7a8a76d2e36349ee5"
format_stamp: "Formatted at 2026-08-07 19:42:24 on dist-test-slave-vq2q"
server_key: "95d5209ab47a40913db3cb176df6bb70"
server_key_iv: "cdc3919304d4669c1674aa843d1feaa2"
server_key_version: "encryptionkey@0"
I20260807 19:42:24.637859 4694 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:42:24.643363 4709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:24.646127 4694 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.007s sys 0.000s
I20260807 19:42:24.646384 4694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "545178215a9a47a7a8a76d2e36349ee5"
format_stamp: "Formatted at 2026-08-07 19:42:24 on dist-test-slave-vq2q"
server_key: "95d5209ab47a40913db3cb176df6bb70"
server_key_iv: "cdc3919304d4669c1674aa843d1feaa2"
server_key_version: "encryptionkey@0"
I20260807 19:42:24.646661 4694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:24.666483 4694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:24.667399 4694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:24.667716 4694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:24.688400 4694 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:37907
I20260807 19:42:24.688416 4762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:37907 every 8 connection(s)
I20260807 19:42:24.689886 4694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:24.692134 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 4694
I20260807 19:42:24.692368 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:42:24.693156 31552 external_mini_cluster.cc:1428] Setting key bfff0ab09e506abb1799e13d47dc915a
I20260807 19:42:24.694519 4764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:24.710080 4764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5: Bootstrap starting.
I20260807 19:42:24.710489 4767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:24.695817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41634 (local address 127.30.208.62:37907)
0807 19:42:24.697300 (+ 1483us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:24.697319 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:24.697370 (+ 51us) server_negotiation.cc:407] Connection header received
0807 19:42:24.699931 (+ 2561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:24.699962 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:24.700533 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:24.701003 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:24.701901 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:24.703279 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:24.705253 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:24.706051 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:24.706208 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:24.707708 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:24.707719 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:24.707739 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:24.708347 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:24.708440 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:24.708464 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:24.708571 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:24.708901 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 19:42:24.709317 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883,"thread_start_us":587,"threads_started":1}
I20260807 19:42:24.714052 4764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:24.715070 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:24.715318 4764 log.cc:824] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:24.718468 4764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5: No bootstrap required, opened a new log
I20260807 19:42:24.726060 4764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545178215a9a47a7a8a76d2e36349ee5" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37907 } }
I20260807 19:42:24.726485 4764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:24.726588 4764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 545178215a9a47a7a8a76d2e36349ee5, State: Initialized, Role: FOLLOWER
I20260807 19:42:24.727210 4764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545178215a9a47a7a8a76d2e36349ee5" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37907 } }
I20260807 19:42:24.727458 4764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:24.727615 4764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:24.727823 4764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:24.730430 4764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545178215a9a47a7a8a76d2e36349ee5" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37907 } }
I20260807 19:42:24.731019 4764 leader_election.cc:302] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 545178215a9a47a7a8a76d2e36349ee5; no voters:
I20260807 19:42:24.731624 4764 leader_election.cc:288] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:24.731762 4769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:24.732446 4769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [term 1 LEADER]: Becoming Leader. State: Replica: 545178215a9a47a7a8a76d2e36349ee5, State: Running, Role: LEADER
I20260807 19:42:24.732973 4769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545178215a9a47a7a8a76d2e36349ee5" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37907 } }
I20260807 19:42:24.733739 4764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:24.735450 4770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "545178215a9a47a7a8a76d2e36349ee5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545178215a9a47a7a8a76d2e36349ee5" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37907 } } }
I20260807 19:42:24.735854 4770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:24.736503 4771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 545178215a9a47a7a8a76d2e36349ee5. Latest consensus state: current_term: 1 leader_uuid: "545178215a9a47a7a8a76d2e36349ee5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "545178215a9a47a7a8a76d2e36349ee5" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37907 } } }
I20260807 19:42:24.736830 4771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:24.738306 4774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:24.743649 4774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:24.753026 4774 catalog_manager.cc:1347] Generated new cluster ID: 888ccbfe0a814caa9d560c8d71e0bbfc
I20260807 19:42:24.753144 4774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:24.767266 4774 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:24.768607 4774 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:24.775914 4774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 545178215a9a47a7a8a76d2e36349ee5: Generated new TSK 0
I20260807 19:42:24.777035 4774 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 19:42:24.834612 4797 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 19:42:24.834883 4797 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 19:42:24.836866 4767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:24.805707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.30.208.62:37907)
0807 19:42:24.805971 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:24.805979 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:24.806441 (+ 462us) server_negotiation.cc:407] Connection header received
0807 19:42:24.806844 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:24.806849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:24.806932 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:24.807079 (+ 147us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:42:24.807721 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:24.808482 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:24.810611 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:24.811098 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:24.811244 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:24.811918 (+ 674us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:42:24.836266 (+ 24348us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43505/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 19:42:24.836599 (+ 333us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41648: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43505/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 19:42:24.837001 4767 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41648: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43505/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 19:42:24.838903 4796 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:24.805559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37907 (local address 127.0.0.1:41648)
0807 19:42:24.806329 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:24.806349 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:24.806550 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:24.807122 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:24.807128 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:24.807142 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:42:24.807542 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:24.807554 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:24.808624 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:24.808629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:24.810436 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:24.810453 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:24.811312 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:24.811319 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:24.811512 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:24.811597 (+ 85us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:42:24.836595 (+ 24998us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43505/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 19:42:24.836990 (+ 395us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.208.62:37907: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43505/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":409,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.208.62:37907: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43505/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 19:42:24.841238 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 4694
I20260807 19:42:24.842415 4757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:24.937321 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 4694
2026-08-07T19:42:24Z chronyd exiting
I20260807 19:42:24.987664 31552 test_util.cc:175] -----------------------------------------------
I20260807 19:42:24.987797 31552 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786131702042267-31552-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (767 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 19:42:24.991612 31552 mini_oidc.cc:152] Starting JWKS server
I20260807 19:42:24.993402 31552 webserver.cc:460] Webserver started at https://127.0.0.1:41653/ using document root <none> and password file <none>
I20260807 19:42:24.993633 31552 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 19:42:24.994258 31552 webserver.cc:460] Webserver started at http://127.0.0.1:46555/ using document root <none> and password file <none>
2026-08-07T19:42:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:25Z Disabled control of system clock
I20260807 19:42:25.026372 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:45213
--webserver_interface=127.30.208.62
--webserver_port=0
--builtin_ntp_servers=127.30.208.20:40053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:45213
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41653/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/testchainca.cert with env {}
W20260807 19:42:25.350237 4805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:25.350550 4805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:25.350692 4805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:25.357992 4805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:25.358112 4805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:25.358179 4805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:25.358263 4805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:25.358323 4805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:42:25.358408 4805 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:42:25.358459 4805 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41653/jwks.json
I20260807 19:42:25.368480 4805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:40053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:45213
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:45213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41653/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:25.369805 4805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:25.371094 4805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:25.382920 4810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:25.382922 4813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:25.382949 4811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:25.383610 4805 server_base.cc:1034] running on GCE node
I20260807 19:42:25.384390 4805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:25.385882 4805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:25.387077 4805 hybrid_clock.cc:648] HybridClock initialized: now 1786131745387042 us; error 28 us; skew 500 ppm
I20260807 19:42:25.387391 4805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:25.393518 4805 webserver.cc:460] Webserver started at http://127.30.208.62:34509/ using document root <none> and password file <none>
I20260807 19:42:25.394228 4805 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:25.394349 4805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:25.394649 4805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:25.397622 4805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "1beed59839224269b8315f15aa64e4ee"
format_stamp: "Formatted at 2026-08-07 19:42:25 on dist-test-slave-vq2q"
server_key: "7034eb62a3f64cb464a98ca6cf8b8a4c"
server_key_iv: "fe33df2dc4cd66daa89f7bf81200e2ef"
server_key_version: "encryptionkey@0"
I20260807 19:42:25.398411 4805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "1beed59839224269b8315f15aa64e4ee"
format_stamp: "Formatted at 2026-08-07 19:42:25 on dist-test-slave-vq2q"
server_key: "7034eb62a3f64cb464a98ca6cf8b8a4c"
server_key_iv: "fe33df2dc4cd66daa89f7bf81200e2ef"
server_key_version: "encryptionkey@0"
I20260807 19:42:25.403443 4805 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:42:25.406841 4820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:25.408116 4805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:42:25.408275 4805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "1beed59839224269b8315f15aa64e4ee"
format_stamp: "Formatted at 2026-08-07 19:42:25 on dist-test-slave-vq2q"
server_key: "7034eb62a3f64cb464a98ca6cf8b8a4c"
server_key_iv: "fe33df2dc4cd66daa89f7bf81200e2ef"
server_key_version: "encryptionkey@0"
I20260807 19:42:25.408463 4805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:25.427166 4805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:25.427987 4805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:25.428275 4805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:25.445426 4805 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:45213
I20260807 19:42:25.445446 4877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:45213 every 8 connection(s)
I20260807 19:42:25.446714 4805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:25.450345 4878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:25.457171 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 4805
I20260807 19:42:25.457448 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:42:25.458254 31552 external_mini_cluster.cc:1428] Setting key 5a1ec14889dc669e4e83a68ce5a1a066
I20260807 19:42:25.461596 4878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee: Bootstrap starting.
I20260807 19:42:25.464614 4878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:25.465652 4878 log.cc:824] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:25.468504 4878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee: No bootstrap required, opened a new log
I20260807 19:42:25.471468 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:25.460183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57442 (local address 127.30.208.62:45213)
0807 19:42:25.461153 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:25.461164 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:25.461231 (+ 67us) server_negotiation.cc:407] Connection header received
0807 19:42:25.462938 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:25.462960 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:25.463416 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:25.463746 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:25.464257 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:25.465600 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:25.467132 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:25.467797 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:25.467966 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:25.469146 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:25.469154 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:25.469168 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:25.469746 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:25.469821 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:25.469846 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:25.469949 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:25.470180 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 19:42:25.470486 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":328,"threads_started":1}
I20260807 19:42:25.475389 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:25.476882 4878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1beed59839224269b8315f15aa64e4ee" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45213 } }
I20260807 19:42:25.477272 4878 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:25.477380 4878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1beed59839224269b8315f15aa64e4ee, State: Initialized, Role: FOLLOWER
I20260807 19:42:25.477993 4878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1beed59839224269b8315f15aa64e4ee" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45213 } }
I20260807 19:42:25.478281 4878 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:25.478427 4878 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:25.478605 4878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:25.480775 4878 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1beed59839224269b8315f15aa64e4ee" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45213 } }
I20260807 19:42:25.481324 4878 leader_election.cc:302] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1beed59839224269b8315f15aa64e4ee; no voters:
I20260807 19:42:25.481901 4878 leader_election.cc:288] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:25.482079 4883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:25.483136 4883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [term 1 LEADER]: Becoming Leader. State: Replica: 1beed59839224269b8315f15aa64e4ee, State: Running, Role: LEADER
I20260807 19:42:25.483657 4883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1beed59839224269b8315f15aa64e4ee" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45213 } }
I20260807 19:42:25.484048 4878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:25.486492 4885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1beed59839224269b8315f15aa64e4ee. Latest consensus state: current_term: 1 leader_uuid: "1beed59839224269b8315f15aa64e4ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1beed59839224269b8315f15aa64e4ee" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45213 } } }
I20260807 19:42:25.486828 4885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:25.487289 4884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1beed59839224269b8315f15aa64e4ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1beed59839224269b8315f15aa64e4ee" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45213 } } }
I20260807 19:42:25.487604 4884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:25.487774 4889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:25.496783 4889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:25.505599 4889 catalog_manager.cc:1347] Generated new cluster ID: f69d56f6368440b0a63ee7c9c61b9e15
I20260807 19:42:25.505735 4889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:25.521310 4889 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:25.523063 4889 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:25.536267 4889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1beed59839224269b8315f15aa64e4ee: Generated new TSK 0
I20260807 19:42:25.537520 4889 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 19:42:25.571059 4912 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 19:42:25.571194 4912 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 19:42:25.572484 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:25.551606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57444 (local address 127.30.208.62:45213)
0807 19:42:25.551805 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:25.551812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:25.552222 (+ 410us) server_negotiation.cc:407] Connection header received
0807 19:42:25.552458 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:25.552463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:25.552576 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:25.552763 (+ 187us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:42:25.553227 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:25.554351 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:25.555766 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:25.556519 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:25.556649 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:25.557352 (+ 703us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:42:25.571763 (+ 14411us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41653/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 19:42:25.572185 (+ 422us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57444: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41653/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 19:42:25.572613 4881 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57444: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41653/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 19:42:25.572623 4911 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:25.551500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:45213 (local address 127.0.0.1:57444)
0807 19:42:25.552115 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:25.552136 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:25.552262 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:25.552765 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:25.552768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:25.552777 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:42:25.553039 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:25.553047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:25.554531 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:25.554535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:25.555607 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:25.555620 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:25.556830 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:25.556835 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:25.556989 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:25.557028 (+ 39us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:42:25.572061 (+ 15033us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41653/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 19:42:25.572279 (+ 218us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.208.62:45213: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41653/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":334,"threads_started":1}
I20260807 19:42:25.574054 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 4805
I20260807 19:42:25.575306 4872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:25.667963 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 4805
2026-08-07T19:42:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (732 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 19:42:25.723881 31552 mini_oidc.cc:152] Starting JWKS server
I20260807 19:42:25.725932 31552 webserver.cc:460] Webserver started at https://127.0.0.1:38353/ using document root <none> and password file <none>
I20260807 19:42:25.726153 31552 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 19:42:25.726827 31552 webserver.cc:460] Webserver started at http://127.0.0.1:33905/ using document root <none> and password file <none>
2026-08-07T19:42:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:25Z Disabled control of system clock
I20260807 19:42:25.758771 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:44493
--webserver_interface=127.30.208.62
--webserver_port=0
--builtin_ntp_servers=127.30.208.20:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:44493
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38353/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 19:42:26.089665 4920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:26.089982 4920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:26.090121 4920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:26.097373 4920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:26.097500 4920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:26.097561 4920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:26.097640 4920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:26.097702 4920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:42:26.097782 4920 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:42:26.097837 4920 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38353/jwks.json
I20260807 19:42:26.107097 4920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:44493
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:44493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38353/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:26.108450 4920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:26.110260 4920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:26.121846 4928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:26.121901 4927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:26.121898 4930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:26.122763 4920 server_base.cc:1034] running on GCE node
I20260807 19:42:26.123668 4920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:26.125514 4920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:26.126817 4920 hybrid_clock.cc:648] HybridClock initialized: now 1786131746126757 us; error 39 us; skew 500 ppm
I20260807 19:42:26.127225 4920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:26.129802 4920 webserver.cc:460] Webserver started at http://127.30.208.62:36627/ using document root <none> and password file <none>
I20260807 19:42:26.130520 4920 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:26.130623 4920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:26.130954 4920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:26.133952 4920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "5807f30c761e4015b806595bfc66b58c"
format_stamp: "Formatted at 2026-08-07 19:42:26 on dist-test-slave-vq2q"
server_key: "fde6ebc6f1c650a52cf982ab70b591d8"
server_key_iv: "8fb9c613302a614a61deb960c1be77c8"
server_key_version: "encryptionkey@0"
I20260807 19:42:26.134738 4920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "5807f30c761e4015b806595bfc66b58c"
format_stamp: "Formatted at 2026-08-07 19:42:26 on dist-test-slave-vq2q"
server_key: "fde6ebc6f1c650a52cf982ab70b591d8"
server_key_iv: "8fb9c613302a614a61deb960c1be77c8"
server_key_version: "encryptionkey@0"
I20260807 19:42:26.139999 4920 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:42:26.143477 4936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:26.144789 4920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 19:42:26.144955 4920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "5807f30c761e4015b806595bfc66b58c"
format_stamp: "Formatted at 2026-08-07 19:42:26 on dist-test-slave-vq2q"
server_key: "fde6ebc6f1c650a52cf982ab70b591d8"
server_key_iv: "8fb9c613302a614a61deb960c1be77c8"
server_key_version: "encryptionkey@0"
I20260807 19:42:26.145157 4920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:26.165839 4920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:26.166707 4920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:26.167008 4920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:26.184684 4920 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:44493
I20260807 19:42:26.184680 4991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:44493 every 8 connection(s)
I20260807 19:42:26.186086 4920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:26.189859 4992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:26.190179 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 4920
I20260807 19:42:26.190385 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:42:26.191200 31552 external_mini_cluster.cc:1428] Setting key d7ccc1ecdbec7a8f06d3a8815a9fbbf2
I20260807 19:42:26.202656 4992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c: Bootstrap starting.
I20260807 19:42:26.204725 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:26.193151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34818 (local address 127.30.208.62:44493)
0807 19:42:26.194162 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:26.194173 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:26.194215 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:42:26.196294 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:26.196319 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:26.196790 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:26.197164 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:26.197705 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:26.198925 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:26.200221 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:26.200921 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:26.201097 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:26.202459 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:26.202467 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:26.202482 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:26.203032 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:26.203115 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:26.203140 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:26.203241 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:26.203468 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 19:42:26.203771 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":335,"threads_started":1}
I20260807 19:42:26.206015 4992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:26.207152 4992 log.cc:824] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:26.208664 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:26.210269 4992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c: No bootstrap required, opened a new log
I20260807 19:42:26.218248 4992 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5807f30c761e4015b806595bfc66b58c" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44493 } }
I20260807 19:42:26.218778 4992 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:26.218895 4992 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5807f30c761e4015b806595bfc66b58c, State: Initialized, Role: FOLLOWER
I20260807 19:42:26.219481 4992 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5807f30c761e4015b806595bfc66b58c" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44493 } }
I20260807 19:42:26.219691 4992 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:26.219801 4992 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:26.219925 4992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:26.222131 4992 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5807f30c761e4015b806595bfc66b58c" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44493 } }
I20260807 19:42:26.222610 4992 leader_election.cc:302] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5807f30c761e4015b806595bfc66b58c; no voters:
I20260807 19:42:26.223188 4992 leader_election.cc:288] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:26.223328 4997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:26.224581 4997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [term 1 LEADER]: Becoming Leader. State: Replica: 5807f30c761e4015b806595bfc66b58c, State: Running, Role: LEADER
I20260807 19:42:26.225111 4997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5807f30c761e4015b806595bfc66b58c" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44493 } }
I20260807 19:42:26.225440 4992 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:26.228144 4999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5807f30c761e4015b806595bfc66b58c. Latest consensus state: current_term: 1 leader_uuid: "5807f30c761e4015b806595bfc66b58c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5807f30c761e4015b806595bfc66b58c" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44493 } } }
I20260807 19:42:26.228574 4999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:26.229046 4998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5807f30c761e4015b806595bfc66b58c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5807f30c761e4015b806595bfc66b58c" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44493 } } }
I20260807 19:42:26.229563 4998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:26.235229 5006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:26.240109 5006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:26.248049 5006 catalog_manager.cc:1347] Generated new cluster ID: 546dbf3daa414a0181498d4530df3078
I20260807 19:42:26.248180 5006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:26.263521 5006 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:26.265400 5006 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:26.283879 5006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5807f30c761e4015b806595bfc66b58c: Generated new TSK 0
I20260807 19:42:26.284806 5006 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:26.312842 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:26.304214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34832 (local address 127.30.208.62:44493)
0807 19:42:26.304432 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:26.304438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:26.304755 (+ 317us) server_negotiation.cc:407] Connection header received
0807 19:42:26.305076 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:26.305081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:26.305168 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:26.305362 (+ 194us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:42:26.305912 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:26.306676 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:26.308744 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:26.309201 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:26.309300 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:26.309919 (+ 619us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:42:26.312126 (+ 2207us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38353/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 19:42:26.312465 (+ 339us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34832: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38353/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":95}
W20260807 19:42:26.312971 4995 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34832: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38353/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 19:42:26.314966 5021 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:26.303987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:44493 (local address 127.0.0.1:34832)
0807 19:42:26.304641 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:26.304662 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:26.304822 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:26.305399 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:26.305405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:26.305418 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:42:26.305747 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:26.305761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:26.306873 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:26.306878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:26.308577 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:26.308594 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:26.309391 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:26.309398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:26.309599 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:26.309647 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:42:26.312352 (+ 2705us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38353/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 19:42:26.314498 (+ 2146us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.208.62:44493: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38353/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":1807,"thread_start_us":346,"threads_started":1}
I20260807 19:42:26.317091 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 4920
I20260807 19:42:26.318377 4987 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 19:42:26.327078 5023 webserver.cc:533] Webserver: sslize failed:
E20260807 19:42:26.327227 5023 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 19:42:26.413779 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 4920
2026-08-07T19:42:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (744 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T19:42:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:26Z Disabled control of system clock
I20260807 19:42:26.498855 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:38741
--webserver_interface=127.30.208.62
--webserver_port=0
--builtin_ntp_servers=127.30.208.20:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:38741
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/testchainca.cert with env {}
W20260807 19:42:26.821789 5028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:26.822115 5028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:26.822214 5028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:26.829443 5028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:26.829561 5028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:26.829605 5028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:26.829663 5028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:26.829713 5028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 19:42:26.829774 5028 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 19:42:26.829813 5028 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 19:42:26.839735 5028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:38741
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:38741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:26.841015 5028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:26.842314 5028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:26.853737 5036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:26.853785 5033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:26.853811 5034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:26.854858 5028 server_base.cc:1034] running on GCE node
I20260807 19:42:26.855686 5028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:26.857095 5028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:26.858314 5028 hybrid_clock.cc:648] HybridClock initialized: now 1786131746858265 us; error 40 us; skew 500 ppm
I20260807 19:42:26.858660 5028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:42:26.860999 5028 webserver.cc:460] Webserver started at http://127.30.208.62:42989/ using document root <none> and password file <none>
I20260807 19:42:26.861712 5028 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:26.861831 5028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:26.862139 5028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:26.865101 5028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "9957fcaade2c410381108b8cb62b5ffc"
format_stamp: "Formatted at 2026-08-07 19:42:26 on dist-test-slave-vq2q"
server_key: "45fdb4e90081d70acb98387b562e57fe"
server_key_iv: "8df9890503f4a463bef93be0f2ce039d"
server_key_version: "encryptionkey@0"
I20260807 19:42:26.865929 5028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "9957fcaade2c410381108b8cb62b5ffc"
format_stamp: "Formatted at 2026-08-07 19:42:26 on dist-test-slave-vq2q"
server_key: "45fdb4e90081d70acb98387b562e57fe"
server_key_iv: "8df9890503f4a463bef93be0f2ce039d"
server_key_version: "encryptionkey@0"
I20260807 19:42:26.871073 5028 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 19:42:26.874508 5042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:26.875903 5028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:26.876083 5028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "9957fcaade2c410381108b8cb62b5ffc"
format_stamp: "Formatted at 2026-08-07 19:42:26 on dist-test-slave-vq2q"
server_key: "45fdb4e90081d70acb98387b562e57fe"
server_key_iv: "8df9890503f4a463bef93be0f2ce039d"
server_key_version: "encryptionkey@0"
I20260807 19:42:26.876295 5028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:26.892947 5028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:26.893819 5028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:26.894068 5028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:26.910301 5028 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:38741
I20260807 19:42:26.910331 5097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:38741 every 8 connection(s)
I20260807 19:42:26.911605 5028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:26.915446 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:26.919121 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 5028
I20260807 19:42:26.919325 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:42:26.919930 31552 external_mini_cluster.cc:1428] Setting key 6fd79ec32aabfd20e1b212517c047dd4
I20260807 19:42:26.928373 5098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc: Bootstrap starting.
I20260807 19:42:26.932040 5098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:26.933198 5098 log.cc:824] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:26.933764 5101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:26.921777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38050 (local address 127.30.208.62:38741)
0807 19:42:26.922767 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:26.922777 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:26.922821 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:42:26.924412 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:26.924436 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:26.924933 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:26.925341 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:26.925941 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:26.927237 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:26.928972 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:26.929715 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:26.929860 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:26.931178 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:26.931201 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:26.931215 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:42:26.931743 (+ 528us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:26.931839 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:42:26.931860 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:26.931954 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:26.932239 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 19:42:26.932693 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":349,"threads_started":1}
I20260807 19:42:26.936339 5098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc: No bootstrap required, opened a new log
I20260807 19:42:26.937685 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:42:26.944136 5098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9957fcaade2c410381108b8cb62b5ffc" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 38741 } }
I20260807 19:42:26.944542 5098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:26.944645 5098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9957fcaade2c410381108b8cb62b5ffc, State: Initialized, Role: FOLLOWER
I20260807 19:42:26.945392 5098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9957fcaade2c410381108b8cb62b5ffc" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 38741 } }
I20260807 19:42:26.945638 5098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:26.945801 5098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:26.946007 5098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:26.948231 5098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9957fcaade2c410381108b8cb62b5ffc" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 38741 } }
I20260807 19:42:26.948729 5098 leader_election.cc:302] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9957fcaade2c410381108b8cb62b5ffc; no voters:
I20260807 19:42:26.949401 5098 leader_election.cc:288] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:26.949563 5103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:26.950311 5103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [term 1 LEADER]: Becoming Leader. State: Replica: 9957fcaade2c410381108b8cb62b5ffc, State: Running, Role: LEADER
I20260807 19:42:26.951017 5103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9957fcaade2c410381108b8cb62b5ffc" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 38741 } }
I20260807 19:42:26.951754 5098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:26.954208 5105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9957fcaade2c410381108b8cb62b5ffc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9957fcaade2c410381108b8cb62b5ffc" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 38741 } } }
I20260807 19:42:26.954175 5106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9957fcaade2c410381108b8cb62b5ffc. Latest consensus state: current_term: 1 leader_uuid: "9957fcaade2c410381108b8cb62b5ffc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9957fcaade2c410381108b8cb62b5ffc" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 38741 } } }
I20260807 19:42:26.954617 5105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:26.954651 5106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:26.955555 5115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:26.960816 5115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:26.969609 5115 catalog_manager.cc:1347] Generated new cluster ID: 670147b565524eae98ddce1e0f3e9b64
I20260807 19:42:26.969748 5115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:26.979734 5115 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:26.981679 5115 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:26.995492 5115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9957fcaade2c410381108b8cb62b5ffc: Generated new TSK 0
I20260807 19:42:26.996589 5115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:27.495183 5101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:27.011408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38066 (local address 127.30.208.62:38741)
0807 19:42:27.011634 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:27.011642 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:27.012118 (+ 476us) server_negotiation.cc:407] Connection header received
0807 19:42:27.012367 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:27.012371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:27.012443 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:27.012586 (+ 143us) server_negotiation.cc:226] Negotiated authn=JWT
0807 19:42:27.013141 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.013913 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.016039 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.016514 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.016628 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.017531 (+ 903us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 19:42:27.494448 (+476917us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 19:42:27.494878 (+ 430us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38066: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 19:42:27.495319 5101 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38066: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 19:42:27.495428 5130 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:27.011348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:38741 (local address 127.0.0.1:38066)
0807 19:42:27.011995 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:27.012016 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:27.012182 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:27.012621 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:27.012626 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:27.012638 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0807 19:42:27.012968 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.012982 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.014063 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.014069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.015836 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:27.015853 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:27.016718 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:27.016726 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:27.017019 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:27.017078 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 19:42:27.494818 (+477740us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 19:42:27.495100 (+ 282us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.208.62:38741: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":281,"threads_started":1}
I20260807 19:42:27.496622 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 5028
I20260807 19:42:27.497795 5093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:27.600033 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 5028
2026-08-07T19:42:27Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1168 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (426 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (420 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:28 dist-test-slave-vq2q krb5kdc[5146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:28 dist-test-slave-vq2q krb5kdc[5146](info): set up 2 sockets
Aug 07 19:42:28 dist-test-slave-vq2q krb5kdc[5146](info): commencing operation
krb5kdc: starting...
W20260807 19:42:29.508635 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.002s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:42:29 dist-test-slave-vq2q krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:42:29.744323 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33555
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:33555
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:30.076767 5159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:30.077080 5159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:30.077270 5159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:30.084621 5159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:30.084818 5159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:30.084903 5159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:30.084988 5159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:30.085049 5159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:30.095116 5159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:33555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:30.096480 5159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:30.097883 5159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:30.109406 5167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:30.109406 5165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:30.109540 5164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:30.110160 5159 server_base.cc:1034] running on GCE node
I20260807 19:42:30.110954 5159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:30.112305 5159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:30.113505 5159 hybrid_clock.cc:648] HybridClock initialized: now 1786131750113455 us; error 41 us; skew 500 ppm
I20260807 19:42:30.113864 5159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:30 dist-test-slave-vq2q krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:30.116961 5159 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:42:30.118569 5159 webserver.cc:460] Webserver started at http://127.30.208.62:35581/ using document root <none> and password file <none>
I20260807 19:42:30.119253 5159 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:30.119374 5159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:30.119678 5159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:30.122722 5159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "17caba580f484573af7bd40e5da54971"
format_stamp: "Formatted at 2026-08-07 19:42:30 on dist-test-slave-vq2q"
server_key: "015c87dc901d994ded699a19d15d33bd"
server_key_iv: "75ffa4ddb35990fa3ff463b5332f3d16"
server_key_version: "encryptionkey@0"
I20260807 19:42:30.123600 5159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "17caba580f484573af7bd40e5da54971"
format_stamp: "Formatted at 2026-08-07 19:42:30 on dist-test-slave-vq2q"
server_key: "015c87dc901d994ded699a19d15d33bd"
server_key_iv: "75ffa4ddb35990fa3ff463b5332f3d16"
server_key_version: "encryptionkey@0"
I20260807 19:42:30.128878 5159 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:42:30.132437 5175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:30.133785 5159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:30.133988 5159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "17caba580f484573af7bd40e5da54971"
format_stamp: "Formatted at 2026-08-07 19:42:30 on dist-test-slave-vq2q"
server_key: "015c87dc901d994ded699a19d15d33bd"
server_key_iv: "75ffa4ddb35990fa3ff463b5332f3d16"
server_key_version: "encryptionkey@0"
I20260807 19:42:30.134188 5159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:30.156175 5159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:30.160692 5159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:30.160977 5159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:30.178938 5159 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:33555
I20260807 19:42:30.178972 5232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:33555 every 8 connection(s)
I20260807 19:42:30.180323 5159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:30.184072 5233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:30.187501 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 5159
I20260807 19:42:30.187736 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:42:30.188514 31552 external_mini_cluster.cc:1428] Setting key 2b76adf6ba37b367c743b033fb771997
I20260807 19:42:30.197039 5233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971: Bootstrap starting.
I20260807 19:42:30.200728 5233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:30.202070 5233 log.cc:824] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:42:30 dist-test-slave-vq2q krb5kdc[5146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:30.205166 5233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971: No bootstrap required, opened a new log
I20260807 19:42:30.211381 5236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.190655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53610 (local address 127.30.208.62:33555)
0807 19:42:30.191404 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:30.191414 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:30.191455 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:42:30.193068 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:30.193089 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:30.193536 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:30.193863 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:30.195525 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.196809 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.198547 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.199224 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.199357 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.204068 (+ 4711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:30.204077 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:30.204091 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:30.204130 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:30.207175 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:30.208012 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:30.208018 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:30.208024 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:30.208191 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:30.208574 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:30.208578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:30.208580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:30.209142 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:30.209432 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:30.209886 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 19:42:30.210276 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":357,"threads_started":1}
I20260807 19:42:30.214071 5233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17caba580f484573af7bd40e5da54971" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33555 } }
I20260807 19:42:30.214525 5233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:30.214610 5233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17caba580f484573af7bd40e5da54971, State: Initialized, Role: FOLLOWER
I20260807 19:42:30.215163 5233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17caba580f484573af7bd40e5da54971" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33555 } }
I20260807 19:42:30.215391 5233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:30.215517 5233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:30.215664 5233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:30.218061 5233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17caba580f484573af7bd40e5da54971" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33555 } }
I20260807 19:42:30.218580 5233 leader_election.cc:302] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17caba580f484573af7bd40e5da54971; no voters:
I20260807 19:42:30.219115 5233 leader_election.cc:288] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:30.219246 5238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:30.219949 5238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [term 1 LEADER]: Becoming Leader. State: Replica: 17caba580f484573af7bd40e5da54971, State: Running, Role: LEADER
I20260807 19:42:30.220593 5238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17caba580f484573af7bd40e5da54971" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33555 } }
I20260807 19:42:30.221326 5233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:30.223361 5239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17caba580f484573af7bd40e5da54971" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17caba580f484573af7bd40e5da54971" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33555 } } }
I20260807 19:42:30.223755 5239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:30.223675 5240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17caba580f484573af7bd40e5da54971. Latest consensus state: current_term: 1 leader_uuid: "17caba580f484573af7bd40e5da54971" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17caba580f484573af7bd40e5da54971" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33555 } } }
I20260807 19:42:30.223965 5240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:30.225363 5246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:30.230916 5246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:30.240180 5246 catalog_manager.cc:1347] Generated new cluster ID: 1262d5f165f8454a8742af16525dc5ef
I20260807 19:42:30.240304 5246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:30.259999 5246 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:30.261399 5246 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:30.269023 5246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17caba580f484573af7bd40e5da54971: Generated new TSK 0
I20260807 19:42:30.269837 5246 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:42:30.328537 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33555
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:30.657649 5262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:30.657938 5262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:30.658035 5262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:30.665351 5262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:30.665473 5262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:30.665596 5262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:42:30.675302 5262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33555
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:30.676697 5262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:30.678045 5262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:30.691054 5267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:30.691004 5268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:30.691093 5270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:30.692019 5262 server_base.cc:1034] running on GCE node
I20260807 19:42:30.692638 5262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:30.693622 5262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:30.694862 5262 hybrid_clock.cc:648] HybridClock initialized: now 1786131750694801 us; error 44 us; skew 500 ppm
I20260807 19:42:30.695215 5262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:30 dist-test-slave-vq2q krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:30.701049 5262 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:42:30.702572 5262 webserver.cc:460] Webserver started at http://127.30.208.1:41203/ using document root <none> and password file <none>
I20260807 19:42:30.703235 5262 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:30.703343 5262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:30.703631 5262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:30.706702 5262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "903cfb030ac64350b82f8225ccb35aa2"
format_stamp: "Formatted at 2026-08-07 19:42:30 on dist-test-slave-vq2q"
server_key: "9f1a7ea8dfb41a46c6a042c0f27ba882"
server_key_iv: "988609286308ff266add2bf5b4aa9a18"
server_key_version: "encryptionkey@0"
I20260807 19:42:30.707461 5262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "903cfb030ac64350b82f8225ccb35aa2"
format_stamp: "Formatted at 2026-08-07 19:42:30 on dist-test-slave-vq2q"
server_key: "9f1a7ea8dfb41a46c6a042c0f27ba882"
server_key_iv: "988609286308ff266add2bf5b4aa9a18"
server_key_version: "encryptionkey@0"
I20260807 19:42:30.712630 5262 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 19:42:30.716107 5277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:30.717461 5262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:30.717617 5262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "903cfb030ac64350b82f8225ccb35aa2"
format_stamp: "Formatted at 2026-08-07 19:42:30 on dist-test-slave-vq2q"
server_key: "9f1a7ea8dfb41a46c6a042c0f27ba882"
server_key_iv: "988609286308ff266add2bf5b4aa9a18"
server_key_version: "encryptionkey@0"
I20260807 19:42:30.717808 5262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:30.732990 5262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:30.737257 5262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:30.737533 5262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:30.738502 5262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:30.739874 5262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:30.739954 5262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:30.740072 5262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:30.740128 5262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:30.770226 5262 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:35101
I20260807 19:42:30.770282 5399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:35101 every 8 connection(s)
I20260807 19:42:30.771521 5262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:30.781343 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 5262
I20260807 19:42:30.781551 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:30.782343 31552 external_mini_cluster.cc:1428] Setting key b5305482f59e306cec8a68ead85182a8
Aug 07 19:42:30 dist-test-slave-vq2q krb5kdc[5146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131750, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:30.791133 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.774010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:56299 (local address 127.30.208.62:33555)
0807 19:42:30.774428 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:30.774432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:30.775286 (+ 854us) server_negotiation.cc:407] Connection header received
0807 19:42:30.777083 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:30.777090 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:30.777190 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:30.777297 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:30.778824 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.779545 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.781192 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.781663 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.781765 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.785651 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:30.785656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:30.785659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:30.785703 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:30.788089 (+ 2386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:30.788795 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:30.788800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:30.788802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:30.788945 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:30.789408 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:30.789412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:30.789414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:30.789675 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:30.789810 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:30.790529 (+ 719us) server_negotiation.cc:299] Negotiation successful
0807 19:42:30.790760 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":214,"threads_started":1}
I20260807 19:42:30.792073 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:30.774437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33555 (local address 127.30.208.1:56299)
0807 19:42:30.775125 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:30.775169 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:30.776813 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:30.777533 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:30.777541 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:30.778060 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:30.778639 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.778657 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.779703 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.779707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.781011 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:30.781029 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:30.781964 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:30.781973 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:30.782308 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:30.783020 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:30.783043 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:30.785448 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:30.788230 (+ 2782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:30.788239 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:30.788257 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:30.788678 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:30.789048 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:30.789053 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:30.789057 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:30.789296 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:30.789798 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:30.789805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:30.790146 (+ 341us) client_negotiation.cc:769] Sending connection context
0807 19:42:30.790405 (+ 259us) client_negotiation.cc:240] Negotiation successful
0807 19:42:30.790939 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":280,"threads_started":1}
I20260807 19:42:30.793550 5400 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33555
I20260807 19:42:30.793857 5400 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:30.794569 5400 heartbeater.cc:511] Master 127.30.208.62:33555 requested a full tablet report, sending...
I20260807 19:42:30.796550 5195 ts_manager.cc:194] Registered new tserver with Master: 903cfb030ac64350b82f8225ccb35aa2 (127.30.208.1:35101)
I20260807 19:42:30.798015 5195 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:56299
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:42:30.840646 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33555
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:31.173494 5406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:31.173791 5406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:31.173928 5406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:31.181479 5406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:31.181622 5406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:31.181780 5406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:42:31.191879 5406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33555
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:31.193156 5406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:31.194453 5406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:31.207154 5414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:31.207273 5413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:31.207154 5417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:31.207926 5406 server_base.cc:1034] running on GCE node
I20260807 19:42:31.208498 5406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:31.209414 5406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:31.210619 5406 hybrid_clock.cc:648] HybridClock initialized: now 1786131751210561 us; error 46 us; skew 500 ppm
I20260807 19:42:31.210958 5406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:31 dist-test-slave-vq2q krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:31.214094 5406 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:42:31.215744 5406 webserver.cc:460] Webserver started at http://127.30.208.2:44549/ using document root <none> and password file <none>
I20260807 19:42:31.216457 5406 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:31.216583 5406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:31.216884 5406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:31.219944 5406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "ea123a6edd84422fbb16a57e0a799a7b"
format_stamp: "Formatted at 2026-08-07 19:42:31 on dist-test-slave-vq2q"
server_key: "98909981ed7bdb4c589791c7598fff48"
server_key_iv: "d53a8fcb61edb2ad90054ab3fa979443"
server_key_version: "encryptionkey@0"
I20260807 19:42:31.220813 5406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "ea123a6edd84422fbb16a57e0a799a7b"
format_stamp: "Formatted at 2026-08-07 19:42:31 on dist-test-slave-vq2q"
server_key: "98909981ed7bdb4c589791c7598fff48"
server_key_iv: "d53a8fcb61edb2ad90054ab3fa979443"
server_key_version: "encryptionkey@0"
I20260807 19:42:31.226804 5406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:42:31.230432 5424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:31.231873 5406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:31.232045 5406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "ea123a6edd84422fbb16a57e0a799a7b"
format_stamp: "Formatted at 2026-08-07 19:42:31 on dist-test-slave-vq2q"
server_key: "98909981ed7bdb4c589791c7598fff48"
server_key_iv: "d53a8fcb61edb2ad90054ab3fa979443"
server_key_version: "encryptionkey@0"
I20260807 19:42:31.232254 5406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:31.262650 5406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:31.267133 5406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:31.267441 5406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:31.268548 5406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:31.270174 5406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:31.270262 5406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:31.270385 5406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:31.270613 5406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:31.301295 5406 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:39557
I20260807 19:42:31.301330 5549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:39557 every 8 connection(s)
I20260807 19:42:31.302608 5406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:31.305852 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 5406
I20260807 19:42:31.306075 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:31.306632 31552 external_mini_cluster.cc:1428] Setting key b2bab3abc751f16672bdbbed73a5d562
Aug 07 19:42:31 dist-test-slave-vq2q krb5kdc[5146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:31.324317 5552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:31.305173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:54229 (local address 127.30.208.62:33555)
0807 19:42:31.305677 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:31.305682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:31.306751 (+ 1069us) server_negotiation.cc:407] Connection header received
0807 19:42:31.308889 (+ 2138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:31.308895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:31.308978 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:31.309071 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:31.310772 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.311536 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.313126 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.313629 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.313747 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:31.317992 (+ 4245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:31.317999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:31.318003 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:31.318041 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:31.320684 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:31.321674 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:31.321679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:31.321681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:31.321832 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:31.322441 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:31.322447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:31.322450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:31.322760 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:31.322889 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:31.323826 (+ 937us) server_negotiation.cc:299] Negotiation successful
0807 19:42:31.324012 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":237,"threads_started":1}
I20260807 19:42:31.325901 5553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:31.305678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33555 (local address 127.30.208.2:54229)
0807 19:42:31.306589 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:31.306629 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:31.308620 (+ 1991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:31.309370 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:31.309379 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:31.309920 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:31.310583 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:31.310604 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.311694 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.311698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:31.312976 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:31.312989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.313870 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.313878 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:31.314110 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:31.314887 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:31.314914 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:31.317669 (+ 2755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:31.320872 (+ 3203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:31.320881 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:31.320899 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:31.321540 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:31.321957 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:31.321962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:31.321966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:31.322246 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:31.322891 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:31.322899 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:31.323355 (+ 456us) client_negotiation.cc:769] Sending connection context
0807 19:42:31.323714 (+ 359us) client_negotiation.cc:240] Negotiation successful
0807 19:42:31.324271 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":409,"threads_started":1}
I20260807 19:42:31.327498 5550 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33555
I20260807 19:42:31.327787 5550 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:31.328516 5550 heartbeater.cc:511] Master 127.30.208.62:33555 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
I20260807 19:42:31.330082 5195 ts_manager.cc:194] Registered new tserver with Master: ea123a6edd84422fbb16a57e0a799a7b (127.30.208.2:39557)
I20260807 19:42:31.330940 5195 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:54229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:42:31.361245 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33555
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:46851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:31.695070 5557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:31.695372 5557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:31.695472 5557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:31.703029 5557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:31.703148 5557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:31.703265 5557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:42:31.713789 5557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:46851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33555
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:31.715118 5557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:31.716449 5557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:31.730090 5563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:31.730125 5562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:31.730271 5566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:31.731010 5557 server_base.cc:1034] running on GCE node
I20260807 19:42:31.731663 5557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:31.732632 5557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:31.733850 5557 hybrid_clock.cc:648] HybridClock initialized: now 1786131751733805 us; error 49 us; skew 500 ppm
I20260807 19:42:31.734192 5557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:31 dist-test-slave-vq2q krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:31.737910 5557 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:42:31.739655 5557 webserver.cc:460] Webserver started at http://127.30.208.3:45497/ using document root <none> and password file <none>
I20260807 19:42:31.740458 5557 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:31.740597 5557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:31.740924 5557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:31.744180 5557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "5454c99081d34aef85c38b0e5b8d908f"
format_stamp: "Formatted at 2026-08-07 19:42:31 on dist-test-slave-vq2q"
server_key: "0a0eb0fcc1549a5859cfec9641a34229"
server_key_iv: "dfbd286b780f3cd0e8921315f49f0b09"
server_key_version: "encryptionkey@0"
I20260807 19:42:31.745042 5557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "5454c99081d34aef85c38b0e5b8d908f"
format_stamp: "Formatted at 2026-08-07 19:42:31 on dist-test-slave-vq2q"
server_key: "0a0eb0fcc1549a5859cfec9641a34229"
server_key_iv: "dfbd286b780f3cd0e8921315f49f0b09"
server_key_version: "encryptionkey@0"
I20260807 19:42:31.750784 5557 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 19:42:31.754657 5573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:31.756120 5557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:31.756295 5557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "5454c99081d34aef85c38b0e5b8d908f"
format_stamp: "Formatted at 2026-08-07 19:42:31 on dist-test-slave-vq2q"
server_key: "0a0eb0fcc1549a5859cfec9641a34229"
server_key_iv: "dfbd286b780f3cd0e8921315f49f0b09"
server_key_version: "encryptionkey@0"
I20260807 19:42:31.756546 5557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:31.774979 5557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:31.780283 5557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:31.780644 5557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:31.783981 5557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:31.785889 5557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:31.785987 5557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:31.786118 5557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:31.786168 5557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:31.800969 5400 heartbeater.cc:503] Master 127.30.208.62:33555 was elected leader, sending a full tablet report...
I20260807 19:42:31.822669 5557 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:36175
I20260807 19:42:31.822747 5696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:36175 every 8 connection(s)
I20260807 19:42:31.824021 5557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:31.825404 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 5557
I20260807 19:42:31.825609 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:31.826030 31552 external_mini_cluster.cc:1428] Setting key 20249ad6eb7eb07273e5c6bc6b896803
Aug 07 19:42:31 dist-test-slave-vq2q krb5kdc[5146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:31.845510 5699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:31.826916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:35715 (local address 127.30.208.62:33555)
0807 19:42:31.827389 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:31.827395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:31.828627 (+ 1232us) server_negotiation.cc:407] Connection header received
0807 19:42:31.830574 (+ 1947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:31.830578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:31.830651 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:31.830756 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:31.832399 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.833288 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.834979 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.835398 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.835503 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:31.839087 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:31.839093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:31.839097 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:31.839136 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:31.842372 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:31.843182 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:31.843187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:31.843189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:31.843322 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:31.843794 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:31.843798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:31.843801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:31.844111 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:31.844254 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:31.844984 (+ 730us) server_negotiation.cc:299] Negotiation successful
0807 19:42:31.845165 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":219,"threads_started":1}
I20260807 19:42:31.846487 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:31.827621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33555 (local address 127.30.208.3:35715)
0807 19:42:31.828466 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:31.828512 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:31.830278 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:31.831044 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:31.831053 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:31.831618 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:31.832217 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:31.832236 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.833463 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.833467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:31.834816 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:31.834830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.835601 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.835606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:31.835824 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:31.836541 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:31.836564 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:31.838846 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:31.842527 (+ 3681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:31.842536 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:31.842550 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:31.843060 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:31.843416 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:31.843419 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:31.843422 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:31.843634 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:31.844223 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:31.844229 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:31.844595 (+ 366us) client_negotiation.cc:769] Sending connection context
0807 19:42:31.844836 (+ 241us) client_negotiation.cc:240] Negotiation successful
0807 19:42:31.845326 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":348,"threads_started":1}
I20260807 19:42:31.847875 5697 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33555
I20260807 19:42:31.848160 5697 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:31.848917 5697 heartbeater.cc:511] Master 127.30.208.62:33555 requested a full tablet report, sending...
I20260807 19:42:31.850379 5195 ts_manager.cc:194] Registered new tserver with Master: 5454c99081d34aef85c38b0e5b8d908f (127.30.208.3:36175)
I20260807 19:42:31.851217 5195 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:35715
I20260807 19:42:31.854162 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/test-admin.keytab.
Aug 07 19:42:31 dist-test-slave-vq2q krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:31.883293 31552 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 19:42:31.888644 5706 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:31.886045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45770 (local address 127.30.208.1:35101)
0807 19:42:31.886706 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:31.886713 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:31.886738 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:42:31.886798 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:31.886804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:31.887021 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:31.887166 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:31.888227 (+ 1061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45770: BlockingRecv error: recv got EOF from 127.0.0.1:45770 (error 108)
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":317,"threads_started":1}
W20260807 19:42:31.889897 31552 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/krb5cc)
Aug 07 19:42:31 dist-test-slave-vq2q krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:31.891587 31552 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 19:42:31 dist-test-slave-vq2q krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:31.893201 31552 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 19:42:31 dist-test-slave-vq2q krb5kdc[5146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:42:31.909412 5706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:31.895163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45786 (local address 127.30.208.1:35101)
0807 19:42:31.895406 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:31.895412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:31.895432 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:42:31.895606 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:31.895612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:31.895675 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:31.895758 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:31.896825 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.897896 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.899188 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.899759 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.899886 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:31.903650 (+ 3764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:31.903665 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:31.903680 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:31.903717 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:31.906516 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:31.907231 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:31.907239 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:31.907241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:31.907391 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:31.907761 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:31.907767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:31.907769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:31.908162 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:31.908464 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:31.908816 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 19:42:31.909090 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260807 19:42:31.915189 5706 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:31.913382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45792 (local address 127.30.208.1:35101)
0807 19:42:31.913550 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:31.913554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:31.913923 (+ 369us) server_negotiation.cc:407] Connection header received
0807 19:42:31.914051 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:31.914055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:31.914124 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:31.914202 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:31.914941 (+ 739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45792: BlockingRecv error: recv got EOF from 127.0.0.1:45792 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 19:42:31.915380 31552 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/krb5cc' of type FILE
Aug 07 19:42:31 dist-test-slave-vq2q krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:31.917009 31552 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 19:42:31 dist-test-slave-vq2q krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:31.918393 31552 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 19:42:31 dist-test-slave-vq2q krb5kdc[5146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:42:31.932775 5706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:31.919753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45796 (local address 127.30.208.1:35101)
0807 19:42:31.919940 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:31.919946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:31.920118 (+ 172us) server_negotiation.cc:407] Connection header received
0807 19:42:31.920281 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:31.920285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:31.920342 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:31.920422 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:31.921502 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.922312 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.923519 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.923948 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.924057 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:31.927836 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:31.927841 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:31.927845 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:31.927880 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:31.930318 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:31.930956 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:31.930960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:31.930963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:31.931090 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:31.931462 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:31.931466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:31.931468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:31.931744 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:31.932008 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:31.932293 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 19:42:31.932502 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 19:42:31.937008 5706 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:31.935388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45806 (local address 127.30.208.1:35101)
0807 19:42:31.935597 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:31.935603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:31.935832 (+ 229us) server_negotiation.cc:407] Connection header received
0807 19:42:31.935959 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:31.935962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:31.936032 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:31.936103 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:31.936790 (+ 687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45806: BlockingRecv error: recv got EOF from 127.0.0.1:45806 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 19:42:31.937147 31552 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786131702042267-31552-0/krb5kdc/krb5cc' of type FILE
Aug 07 19:42:31 dist-test-slave-vq2q krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:31.938752 31552 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 19:42:31 dist-test-slave-vq2q krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:31.940053 31552 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 19:42:31 dist-test-slave-vq2q krb5kdc[5146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:42:31.954566 5706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:31.941419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45814 (local address 127.30.208.1:35101)
0807 19:42:31.941623 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:31.941628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:31.941853 (+ 225us) server_negotiation.cc:407] Connection header received
0807 19:42:31.941980 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:31.941983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:31.942043 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:31.942122 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:31.943077 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.943791 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.945026 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.945554 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.945672 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:31.949348 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:31.949353 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:31.949356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:31.949393 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:31.951850 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:31.952549 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:31.952554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:31.952556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:31.952717 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:31.953102 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:31.953105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:31.953108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:31.953463 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:31.953701 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:31.954043 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 19:42:31.954228 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:42:31.956079 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 5262
I20260807 19:42:31.967674 5706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:31.956772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45818 (local address 127.30.208.1:35101)
0807 19:42:31.956939 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:31.956944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:31.957253 (+ 309us) server_negotiation.cc:407] Connection header received
0807 19:42:31.957423 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:31.957426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:31.957483 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:31.957558 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:31.958476 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.959196 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.960361 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:31.960866 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:31.960973 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:31.963035 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:31.963040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:31.963043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:31.963073 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:31.965345 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:31.965967 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:31.965971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:31.965973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:31.966108 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:31.966471 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:31.966475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:31.966478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:31.966750 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:31.966955 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:31.967231 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 19:42:31.967411 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 19:42:31.968297 5393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:32.054355 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 5262
I20260807 19:42:32.071183 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 5406
Aug 07 19:42:32 dist-test-slave-vq2q krb5kdc[5146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:42:32.086823 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:32.072162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53464 (local address 127.30.208.2:39557)
0807 19:42:32.072662 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:32.072669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:32.072694 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:42:32.072752 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:32.072758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:32.072976 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:32.073130 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:32.074138 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:32.075144 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:32.076381 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:32.076929 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:32.077048 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:32.080938 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:32.080951 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:32.080966 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:32.081002 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:32.083836 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:32.084573 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:32.084581 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:32.084583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:32.084714 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:32.085113 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:32.085117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:32.085119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:32.085542 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:32.085853 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:32.086209 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 19:42:32.086483 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":255,"threads_started":1}
I20260807 19:42:32.088104 5545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:32.173209 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 5406
I20260807 19:42:32.193974 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 5557
Aug 07 19:42:32 dist-test-slave-vq2q krb5kdc[5146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:42:32.210322 5722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:32.195122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60954 (local address 127.30.208.3:36175)
0807 19:42:32.195801 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:32.195808 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:32.195837 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:42:32.195898 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:32.195904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:32.196114 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:32.196251 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:32.197291 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:32.198314 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:32.199594 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:32.200225 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:32.200333 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:32.204642 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:32.204656 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:32.204671 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:32.204710 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:32.207582 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:32.208306 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:32.208313 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:32.208316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:32.208462 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:32.208759 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:32.208761 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:32.208763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:32.209133 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:32.209473 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:32.209757 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 19:42:32.210000 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":362,"threads_started":1}
I20260807 19:42:32.211637 5692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:32.295331 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 5557
I20260807 19:42:32.311873 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 5159
I20260807 19:42:32.312978 5227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:32.416941 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 5159
2026-08-07T19:42:32Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3972 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:32 dist-test-slave-vq2q krb5kdc[5729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:32 dist-test-slave-vq2q krb5kdc[5729](info): set up 2 sockets
Aug 07 19:42:32 dist-test-slave-vq2q krb5kdc[5729](info): commencing operation
krb5kdc: starting...
W20260807 19:42:33.484607 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.002s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:42:33 dist-test-slave-vq2q krb5kdc[5729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:33Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.62.keytab.
Entry for principal oryx/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.62.keytab.
I20260807 19:42:33.721098 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33613
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.62.keytab
--principal=oryx/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:33613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:34.046803 5743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:34.047103 5743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:34.047240 5743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:34.054648 5743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:34.054752 5743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:34.054832 5743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:34.054909 5743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:34.054970 5743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:34.064878 5743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:33613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.62.keytab
--principal=oryx/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:34.066243 5743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:34.067495 5743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:34.078905 5749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:34.078943 5750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:34.079010 5752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:34.079674 5743 server_base.cc:1034] running on GCE node
I20260807 19:42:34.080483 5743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:34.081998 5743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:34.083209 5743 hybrid_clock.cc:648] HybridClock initialized: now 1786131754083152 us; error 49 us; skew 500 ppm
I20260807 19:42:34.083554 5743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:34 dist-test-slave-vq2q krb5kdc[5729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131754, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:34.086673 5743 init.cc:377] Logged in from keytab as oryx/127.30.208.62@KRBTEST.COM (short username oryx)
I20260807 19:42:34.088130 5743 webserver.cc:460] Webserver started at http://127.30.208.62:45559/ using document root <none> and password file <none>
I20260807 19:42:34.088800 5743 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:34.088922 5743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:34.089241 5743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:34.092193 5743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "4b72ec6924c24c76b5bee353ab72410d"
format_stamp: "Formatted at 2026-08-07 19:42:34 on dist-test-slave-vq2q"
server_key: "3b579f6a1b579a7ef27d66b30f8286f9"
server_key_iv: "ca3d943224151a209fe8170eff8a5758"
server_key_version: "encryptionkey@0"
I20260807 19:42:34.092947 5743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "4b72ec6924c24c76b5bee353ab72410d"
format_stamp: "Formatted at 2026-08-07 19:42:34 on dist-test-slave-vq2q"
server_key: "3b579f6a1b579a7ef27d66b30f8286f9"
server_key_iv: "ca3d943224151a209fe8170eff8a5758"
server_key_version: "encryptionkey@0"
I20260807 19:42:34.098124 5743 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260807 19:42:34.101486 5759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:34.102861 5743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:42:34.103031 5743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "4b72ec6924c24c76b5bee353ab72410d"
format_stamp: "Formatted at 2026-08-07 19:42:34 on dist-test-slave-vq2q"
server_key: "3b579f6a1b579a7ef27d66b30f8286f9"
server_key_iv: "ca3d943224151a209fe8170eff8a5758"
server_key_version: "encryptionkey@0"
I20260807 19:42:34.103222 5743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:34.128204 5743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:34.132467 5743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:34.132768 5743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:34.150359 5743 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:33613
I20260807 19:42:34.150403 5813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:33613 every 8 connection(s)
I20260807 19:42:34.152048 5743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:34.154219 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 5743
I20260807 19:42:34.154456 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:42:34.155105 31552 external_mini_cluster.cc:1428] Setting key 117db540317db054d8574c9925a8acd3
I20260807 19:42:34.155710 5814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:34.169601 5814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d: Bootstrap starting.
Aug 07 19:42:34 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
I20260807 19:42:34.173411 5814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:34.174777 5814 log.cc:824] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:34.178367 5814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d: No bootstrap required, opened a new log
I20260807 19:42:34.179664 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:34.156972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55524 (local address 127.30.208.62:33613)
0807 19:42:34.157830 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:34.157845 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:34.157894 (+ 49us) server_negotiation.cc:407] Connection header received
0807 19:42:34.160316 (+ 2422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:34.160347 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:34.160843 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:34.161304 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:34.162642 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:34.164556 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:34.165860 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:34.166908 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:34.167088 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:34.170990 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:34.171001 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:34.171025 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:34.171081 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:34.174990 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:34.175913 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:34.175919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:34.175925 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:34.176102 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:34.176487 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:34.176492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:34.176495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:34.177171 (+ 676us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:34.177525 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:34.177887 (+ 362us) server_negotiation.cc:299] Negotiation successful
0807 19:42:34.178321 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":344,"threads_started":1}
I20260807 19:42:34.187088 5814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } }
I20260807 19:42:34.187539 5814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:34.187628 5814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b72ec6924c24c76b5bee353ab72410d, State: Initialized, Role: FOLLOWER
I20260807 19:42:34.188162 5814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } }
I20260807 19:42:34.188392 5814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:34.188495 5814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:34.188621 5814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:34.190858 5814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } }
I20260807 19:42:34.191324 5814 leader_election.cc:302] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b72ec6924c24c76b5bee353ab72410d; no voters:
I20260807 19:42:34.191934 5814 leader_election.cc:288] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:34.192080 5820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:34.192977 5820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [term 1 LEADER]: Becoming Leader. State: Replica: 4b72ec6924c24c76b5bee353ab72410d, State: Running, Role: LEADER
I20260807 19:42:34.193578 5820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } }
I20260807 19:42:34.194217 5814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:34.196216 5822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b72ec6924c24c76b5bee353ab72410d. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } }
I20260807 19:42:34.197221 5822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:34.196859 5821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } }
I20260807 19:42:34.197714 5821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:34.202642 5833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:34.208272 5833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:34.216223 5833 catalog_manager.cc:1347] Generated new cluster ID: 48ade388e5c844bf927e36d1c2fe5952
I20260807 19:42:34.216356 5833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:34.232167 5833 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:34.234115 5833 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:34.243089 5833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d: Generated new TSK 0
I20260807 19:42:34.244174 5833 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.1.keytab.
Entry for principal oryx/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.1.keytab.
I20260807 19:42:34.312842 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33613
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.1.keytab
--principal=oryx/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:34.641923 5845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:34.642225 5845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:34.642350 5845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:34.649701 5845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:34.649822 5845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:34.649977 5845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:42:34.659835 5845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.1.keytab
--principal=oryx/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:34.661171 5845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:34.662518 5845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:34.675118 5850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:34.675498 5853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:34.675635 5851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:34.676177 5845 server_base.cc:1034] running on GCE node
I20260807 19:42:34.676869 5845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:34.677835 5845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:34.679044 5845 hybrid_clock.cc:648] HybridClock initialized: now 1786131754678996 us; error 43 us; skew 500 ppm
I20260807 19:42:34.679371 5845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:34 dist-test-slave-vq2q krb5kdc[5729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131754, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:34.682538 5845 init.cc:377] Logged in from keytab as oryx/127.30.208.1@KRBTEST.COM (short username oryx)
I20260807 19:42:34.683928 5845 webserver.cc:460] Webserver started at http://127.30.208.1:39937/ using document root <none> and password file <none>
I20260807 19:42:34.684607 5845 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:34.684728 5845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:34.685029 5845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:34.688019 5845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee"
format_stamp: "Formatted at 2026-08-07 19:42:34 on dist-test-slave-vq2q"
server_key: "fedbbb89f8d40355777362095057f92a"
server_key_iv: "d9f4ab4706f32762d97963a92cbc9be0"
server_key_version: "encryptionkey@0"
I20260807 19:42:34.688791 5845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee"
format_stamp: "Formatted at 2026-08-07 19:42:34 on dist-test-slave-vq2q"
server_key: "fedbbb89f8d40355777362095057f92a"
server_key_iv: "d9f4ab4706f32762d97963a92cbc9be0"
server_key_version: "encryptionkey@0"
I20260807 19:42:34.694188 5845 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 19:42:34.697896 5861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:34.699384 5845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 19:42:34.699549 5845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee"
format_stamp: "Formatted at 2026-08-07 19:42:34 on dist-test-slave-vq2q"
server_key: "fedbbb89f8d40355777362095057f92a"
server_key_iv: "d9f4ab4706f32762d97963a92cbc9be0"
server_key_version: "encryptionkey@0"
I20260807 19:42:34.699736 5845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:34.720957 5845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:34.725605 5845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:34.725903 5845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:34.726913 5845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:34.728300 5845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:34.728380 5845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:34.728499 5845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:34.728555 5845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:34.759851 5845 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:35269
I20260807 19:42:34.759899 5982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:35269 every 8 connection(s)
I20260807 19:42:34.761119 5845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:34.766681 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 5845
I20260807 19:42:34.766902 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:34.767501 31552 external_mini_cluster.cc:1428] Setting key d4f191a3d2fe297f5d5948237a7dd300
Aug 07 19:42:34 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131754, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
I20260807 19:42:34.781124 5987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:34.763729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:53461 (local address 127.30.208.62:33613)
0807 19:42:34.764172 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:34.764177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:34.765062 (+ 885us) server_negotiation.cc:407] Connection header received
0807 19:42:34.766984 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:34.766990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:34.767073 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:34.767167 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:34.768694 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:34.769496 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:34.771038 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:34.771435 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:34.771555 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:34.775403 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:34.775408 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:34.775411 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:34.775446 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:34.777936 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:34.778720 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:34.778725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:34.778728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:34.778877 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:34.779267 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:34.779272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:34.779275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:34.779617 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:34.779764 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:34.780542 (+ 778us) server_negotiation.cc:299] Negotiation successful
0807 19:42:34.780797 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":213,"threads_started":1}
I20260807 19:42:34.782047 5988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:34.764209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.1:53461)
0807 19:42:34.764907 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:34.764944 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:34.766686 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:34.767422 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:34.767430 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:34.767965 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:34.768516 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:34.768533 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:34.769643 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:34.769647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:34.770886 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:34.770899 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:34.771659 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:34.771667 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:34.771921 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:34.772578 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:34.772602 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:34.775155 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:34.778105 (+ 2950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:34.778113 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:34.778127 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:34.778601 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:34.778976 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:34.778979 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:34.778981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:34.779163 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:34.779772 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:34.779779 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:34.780149 (+ 370us) client_negotiation.cc:769] Sending connection context
0807 19:42:34.780403 (+ 254us) client_negotiation.cc:240] Negotiation successful
0807 19:42:34.780864 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":262,"threads_started":1}
I20260807 19:42:34.783586 5983 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33613
I20260807 19:42:34.783886 5983 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:34.784621 5983 heartbeater.cc:511] Master 127.30.208.62:33613 requested a full tablet report, sending...
I20260807 19:42:34.786901 5777 ts_manager.cc:194] Registered new tserver with Master: 3a6ac8ef1cb744cbbf4d3916bf4d44ee (127.30.208.1:35269)
I20260807 19:42:34.788395 5777 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:53461
WARNING: no policy specified for oryx/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.2.keytab.
Entry for principal oryx/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.2.keytab.
I20260807 19:42:34.820917 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33613
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.2.keytab
--principal=oryx/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:35.148063 5991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:35.148370 5991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:35.148518 5991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:35.156857 5991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:35.156985 5991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:35.157140 5991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:42:35.167070 5991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.2.keytab
--principal=oryx/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:35.168298 5991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:35.169628 5991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:35.182404 5997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:35.182468 5996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:35.182448 5999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:35.183233 5991 server_base.cc:1034] running on GCE node
I20260807 19:42:35.183832 5991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:35.184708 5991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:35.185930 5991 hybrid_clock.cc:648] HybridClock initialized: now 1786131755185876 us; error 44 us; skew 500 ppm
I20260807 19:42:35.186244 5991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:35 dist-test-slave-vq2q krb5kdc[5729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131755, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:35.189410 5991 init.cc:377] Logged in from keytab as oryx/127.30.208.2@KRBTEST.COM (short username oryx)
I20260807 19:42:35.190866 5991 webserver.cc:460] Webserver started at http://127.30.208.2:45631/ using document root <none> and password file <none>
I20260807 19:42:35.191535 5991 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:35.191655 5991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:35.191953 5991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:35.195003 5991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "ca911cece32a4d55ba20d1bb25ac8deb"
format_stamp: "Formatted at 2026-08-07 19:42:35 on dist-test-slave-vq2q"
server_key: "dff10d9a2f538e81c08ad07d04d3455c"
server_key_iv: "dda851c193cceedf31cb2024db33b748"
server_key_version: "encryptionkey@0"
I20260807 19:42:35.195829 5991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "ca911cece32a4d55ba20d1bb25ac8deb"
format_stamp: "Formatted at 2026-08-07 19:42:35 on dist-test-slave-vq2q"
server_key: "dff10d9a2f538e81c08ad07d04d3455c"
server_key_iv: "dda851c193cceedf31cb2024db33b748"
server_key_version: "encryptionkey@0"
I20260807 19:42:35.201122 5991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 19:42:35.204818 6008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:35.206254 5991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:35.206418 5991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "ca911cece32a4d55ba20d1bb25ac8deb"
format_stamp: "Formatted at 2026-08-07 19:42:35 on dist-test-slave-vq2q"
server_key: "dff10d9a2f538e81c08ad07d04d3455c"
server_key_iv: "dda851c193cceedf31cb2024db33b748"
server_key_version: "encryptionkey@0"
I20260807 19:42:35.206610 5991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:35.255010 5991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:35.259298 5991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:35.259579 5991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:35.260589 5991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:35.262049 5991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:35.262130 5991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:35.262256 5991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:35.262310 5991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:35.293927 5991 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:38995
I20260807 19:42:35.293951 6131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:38995 every 8 connection(s)
I20260807 19:42:35.295226 5991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:35.295501 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 5991
I20260807 19:42:35.295689 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:35.296206 31552 external_mini_cluster.cc:1428] Setting key f5db27b00579a4abeaa0fa572ef96f76
Aug 07 19:42:35 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131755, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
I20260807 19:42:35.317009 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.298736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:60299 (local address 127.30.208.62:33613)
0807 19:42:35.299146 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:35.299150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:35.300366 (+ 1216us) server_negotiation.cc:407] Connection header received
0807 19:42:35.302261 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:35.302267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:35.302356 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:35.302453 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:35.304071 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.304897 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.306530 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.306955 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.307089 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.310632 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:35.310639 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:35.310643 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:35.310684 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:35.313798 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:35.314589 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:35.314594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:35.314596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:35.314744 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:35.315137 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:35.315141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:35.315144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:35.315481 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:35.315636 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:35.316398 (+ 762us) server_negotiation.cc:299] Negotiation successful
0807 19:42:35.316622 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":212,"threads_started":1}
I20260807 19:42:35.317909 6135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.299464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.2:60299)
0807 19:42:35.300203 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:35.300253 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:35.301996 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:35.302707 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:35.302716 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:35.303304 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:35.303903 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.303921 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.305054 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.305058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.306393 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.306406 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.307195 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.307203 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.307487 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.308200 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:35.308225 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:35.310385 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:35.313975 (+ 3590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:35.313984 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:35.313997 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:35.314447 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:35.314840 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:35.314844 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:35.314846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:35.315029 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:35.315631 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:35.315638 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:35.316020 (+ 382us) client_negotiation.cc:769] Sending connection context
0807 19:42:35.316269 (+ 249us) client_negotiation.cc:240] Negotiation successful
0807 19:42:35.316688 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":307,"threads_started":1}
I20260807 19:42:35.319339 6132 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33613
I20260807 19:42:35.319677 6132 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:35.320618 6132 heartbeater.cc:511] Master 127.30.208.62:33613 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.30.208.3@KRBTEST.COM; defaulting to no policy
I20260807 19:42:35.322324 5777 ts_manager.cc:194] Registered new tserver with Master: ca911cece32a4d55ba20d1bb25ac8deb (127.30.208.2:38995)
I20260807 19:42:35.323226 5777 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:60299
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.3.keytab.
Entry for principal oryx/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.3.keytab.
I20260807 19:42:35.349958 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33613
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.3.keytab
--principal=oryx/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:35.677338 6138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:35.677671 6138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:35.677824 6138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:35.685343 6138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:35.685482 6138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:35.685652 6138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:42:35.695874 6138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.3.keytab
--principal=oryx/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:35.697330 6138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:35.698701 6138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:35.712337 6144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:35.712320 6146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:35.712561 6143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:35.713244 6138 server_base.cc:1034] running on GCE node
I20260807 19:42:35.713873 6138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:35.714831 6138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:35.716040 6138 hybrid_clock.cc:648] HybridClock initialized: now 1786131755715983 us; error 47 us; skew 500 ppm
I20260807 19:42:35.716367 6138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:35 dist-test-slave-vq2q krb5kdc[5729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131755, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:35.723073 6138 init.cc:377] Logged in from keytab as oryx/127.30.208.3@KRBTEST.COM (short username oryx)
I20260807 19:42:35.724591 6138 webserver.cc:460] Webserver started at http://127.30.208.3:38615/ using document root <none> and password file <none>
I20260807 19:42:35.725309 6138 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:35.725422 6138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:35.725696 6138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:35.728626 6138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "132385f9496d4ecbb98ec6d6419b46dd"
format_stamp: "Formatted at 2026-08-07 19:42:35 on dist-test-slave-vq2q"
server_key: "e6882ec753f670d51ed3ab86dda9bbec"
server_key_iv: "340a3c8f0339e26be4eaf8ede488eeef"
server_key_version: "encryptionkey@0"
I20260807 19:42:35.729401 6138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "132385f9496d4ecbb98ec6d6419b46dd"
format_stamp: "Formatted at 2026-08-07 19:42:35 on dist-test-slave-vq2q"
server_key: "e6882ec753f670d51ed3ab86dda9bbec"
server_key_iv: "340a3c8f0339e26be4eaf8ede488eeef"
server_key_version: "encryptionkey@0"
I20260807 19:42:35.734573 6138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:42:35.738014 6153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:35.739336 6138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:35.739481 6138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "132385f9496d4ecbb98ec6d6419b46dd"
format_stamp: "Formatted at 2026-08-07 19:42:35 on dist-test-slave-vq2q"
server_key: "e6882ec753f670d51ed3ab86dda9bbec"
server_key_iv: "340a3c8f0339e26be4eaf8ede488eeef"
server_key_version: "encryptionkey@0"
I20260807 19:42:35.739672 6138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:35.757319 6138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:35.762106 6138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:35.762446 6138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:35.763608 6138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:35.765323 6138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:35.765415 6138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:35.765537 6138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:35.765596 6138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:35.792094 5983 heartbeater.cc:503] Master 127.30.208.62:33613 was elected leader, sending a full tablet report...
I20260807 19:42:35.801226 6138 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:39325
I20260807 19:42:35.801286 6273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:39325 every 8 connection(s)
I20260807 19:42:35.802488 6138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:35.802778 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 6138
I20260807 19:42:35.802977 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:35.803582 31552 external_mini_cluster.cc:1428] Setting key cca204ed79dc5aff34f981acf78391c6
Aug 07 19:42:35 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131755, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
I20260807 19:42:35.822932 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.805121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:33733 (local address 127.30.208.62:33613)
0807 19:42:35.805328 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:35.805334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:35.806815 (+ 1481us) server_negotiation.cc:407] Connection header received
0807 19:42:35.808719 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:35.808724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:35.808830 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:35.808916 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:35.810563 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.811480 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.813058 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.813521 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.813638 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.817158 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:35.817163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:35.817167 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:35.817241 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:35.820199 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:35.820914 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:35.820918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:35.820920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:35.821037 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:35.821465 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:35.821468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:35.821470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:35.821732 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:35.821864 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:35.822533 (+ 669us) server_negotiation.cc:299] Negotiation successful
0807 19:42:35.822701 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:42:35.823901 6276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:35.805840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.3:33733)
0807 19:42:35.806665 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:35.806702 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:35.808465 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:35.809168 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:35.809206 (+ 38us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:35.809744 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:35.810399 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.810422 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.811652 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.811658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.812915 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:35.812929 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:35.813765 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:35.813773 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:35.814024 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:35.814684 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:35.814714 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:35.816940 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:35.820334 (+ 3394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:35.820342 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:35.820357 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:35.820822 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:35.821115 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:35.821118 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:35.821120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:35.821370 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:35.821848 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:35.821854 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:35.822215 (+ 361us) client_negotiation.cc:769] Sending connection context
0807 19:42:35.822452 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 19:42:35.822853 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":389,"threads_started":1}
I20260807 19:42:35.825171 6274 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33613
I20260807 19:42:35.825495 6274 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:35.826228 6274 heartbeater.cc:511] Master 127.30.208.62:33613 requested a full tablet report, sending...
I20260807 19:42:35.827630 5777 ts_manager.cc:194] Registered new tserver with Master: 132385f9496d4ecbb98ec6d6419b46dd (127.30.208.3:39325)
I20260807 19:42:35.828418 5777 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:33733
I20260807 19:42:35.832072 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.208.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.208.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.208.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.61.keytab.
Entry for principal oryx/127.30.208.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.61.keytab.
I20260807 19:42:35.885315 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.61:40519
--webserver_interface=127.30.208.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.61.keytab
--principal=oryx/127.30.208.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:33613,127.30.208.61:40519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:36.218842 6279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:36.219130 6279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:36.219260 6279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:36.226523 6279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:36.226637 6279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:36.226711 6279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:36.226785 6279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:36.226851 6279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:36.236765 6279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:33613,127.30.208.61:40519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.61.keytab
--principal=oryx/127.30.208.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.61:40519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.208.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
W20260807 19:42:36.237164 6279 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.208.62:33613,127.30.208.61:40519'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 19:42:36.238226 6279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:36.239521 6279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:36.250900 6285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:36.250969 6288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:36.250998 6284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:36.251729 6279 server_base.cc:1034] running on GCE node
I20260807 19:42:36.252480 6279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:36.253921 6279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:36.255128 6279 hybrid_clock.cc:648] HybridClock initialized: now 1786131756255067 us; error 33 us; skew 500 ppm
I20260807 19:42:36.255440 6279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:36 dist-test-slave-vq2q krb5kdc[5729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131756, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:36.258474 6279 init.cc:377] Logged in from keytab as oryx/127.30.208.61@KRBTEST.COM (short username oryx)
I20260807 19:42:36.259931 6279 webserver.cc:460] Webserver started at http://127.30.208.61:42931/ using document root <none> and password file <none>
I20260807 19:42:36.260614 6279 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:36.260705 6279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:36.261041 6279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:36.264104 6279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/data/instance:
uuid: "f1aaac13b4e84290b4e10e03dbafc748"
format_stamp: "Formatted at 2026-08-07 19:42:36 on dist-test-slave-vq2q"
server_key: "20d6c7c82dd78ccc08beefe9cbfe9d59"
server_key_iv: "2cc5fabef4d8f851437c648607462679"
server_key_version: "encryptionkey@0"
I20260807 19:42:36.264865 6279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/wal/instance:
uuid: "f1aaac13b4e84290b4e10e03dbafc748"
format_stamp: "Formatted at 2026-08-07 19:42:36 on dist-test-slave-vq2q"
server_key: "20d6c7c82dd78ccc08beefe9cbfe9d59"
server_key_iv: "2cc5fabef4d8f851437c648607462679"
server_key_version: "encryptionkey@0"
I20260807 19:42:36.269964 6279 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260807 19:42:36.273393 6297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:36.274842 6279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:42:36.275003 6279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/wal
uuid: "f1aaac13b4e84290b4e10e03dbafc748"
format_stamp: "Formatted at 2026-08-07 19:42:36 on dist-test-slave-vq2q"
server_key: "20d6c7c82dd78ccc08beefe9cbfe9d59"
server_key_iv: "2cc5fabef4d8f851437c648607462679"
server_key_version: "encryptionkey@0"
I20260807 19:42:36.275202 6279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:36.297108 6279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:36.301252 6279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:36.301474 6279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:36.318786 6279 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.61:40519
I20260807 19:42:36.318809 6354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.61:40519 every 8 connection(s)
I20260807 19:42:36.320092 6279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/data/info.pb
I20260807 19:42:36.323721 6355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:36.325898 6132 heartbeater.cc:503] Master 127.30.208.62:33613 was elected leader, sending a full tablet report...
I20260807 19:42:36.329121 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 6279
I20260807 19:42:36.330705 6355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:42:36 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131756, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.61@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
Aug 07 19:42:36 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.208.61@KRBTEST.COM
I20260807 19:42:36.350737 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.333226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55528 (local address 127.30.208.62:33613)
0807 19:42:36.333674 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:36.333679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:36.334234 (+ 555us) server_negotiation.cc:407] Connection header received
0807 19:42:36.334504 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:36.334509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:36.334591 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:36.334699 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:36.336271 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.337467 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.339469 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.340081 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.340206 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.344331 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:36.344338 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:36.344342 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:36.344385 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:36.347114 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.348570 (+ 1456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.348574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.348576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.348692 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.349060 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.349065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.349068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.349420 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:36.349544 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:36.350147 (+ 603us) server_negotiation.cc:299] Negotiation successful
0807 19:42:36.350387 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":220,"threads_started":1}
I20260807 19:42:36.351619 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.331202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59100 (local address 127.30.208.61:40519)
0807 19:42:36.331964 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:36.331975 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:36.332015 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:42:36.333892 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:36.333914 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:36.334282 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:36.334524 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:36.336269 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.337872 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.339518 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.340315 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.340456 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.344990 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:36.344999 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:36.345011 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:36.345043 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:36.347945 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.348552 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.348559 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.348561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.348702 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.349082 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.349085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.349087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.349645 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:36.349945 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:36.350167 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 19:42:36.350490 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":557,"thread_start_us":338,"threads_started":1}
I20260807 19:42:36.351655 6361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.333198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:55528)
0807 19:42:36.334083 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:36.334129 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:36.334287 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:36.334790 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:36.334797 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:36.335599 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:36.336118 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.336141 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.337593 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.337596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.339316 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.339332 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.340303 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.340309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.340460 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.341364 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:36.341398 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:36.344144 (+ 2746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:36.347244 (+ 3100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:36.347255 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:36.347268 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:36.347964 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:36.348796 (+ 832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:36.348801 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:36.348805 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:36.348956 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:36.349540 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:36.349547 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:36.349759 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 19:42:36.350055 (+ 296us) client_negotiation.cc:240] Negotiation successful
0807 19:42:36.350707 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":215,"thread_start_us":467,"threads_started":1}
I20260807 19:42:36.353632 6355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:42:36 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131756, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.61@KRBTEST.COM for oryx/127.30.208.61@KRBTEST.COM
I20260807 19:42:36.364691 6361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.354452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:40519 (local address 127.0.0.1:59108)
0807 19:42:36.354602 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:36.354620 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:36.354751 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:36.355013 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:36.355017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:36.355342 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:36.355461 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.355469 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.356401 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.356405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.357295 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.357307 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.357923 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.357929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.358110 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.358539 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:36.358554 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:36.360391 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:36.362874 (+ 2483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:36.362878 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:36.362881 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:36.363212 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:36.363505 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:36.363508 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:36.363511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:36.363634 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:36.364062 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:36.364065 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:36.364173 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 19:42:36.364272 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 19:42:36.364466 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 19:42:36.364868 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.354638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59108 (local address 127.30.208.61:40519)
0807 19:42:36.354787 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:36.354792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:36.354807 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:42:36.354845 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:36.354850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:36.354906 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:36.354975 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:36.355581 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.356269 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.357418 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.357798 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.357904 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.360543 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:36.360548 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:36.360551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:36.360579 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:36.362737 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.363303 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.363306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.363309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.363411 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.363717 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.363720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.363722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.363973 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:36.364077 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:36.364341 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 19:42:36.364585 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":45}
I20260807 19:42:36.369161 6355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: Bootstrap starting.
I20260807 19:42:36.372116 6355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:36.373112 6355 log.cc:824] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:36.375873 6355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: No bootstrap required, opened a new log
I20260807 19:42:36.383141 6355 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } }
I20260807 19:42:36.383464 6355 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:36.383558 6355 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1aaac13b4e84290b4e10e03dbafc748, State: Initialized, Role: FOLLOWER
I20260807 19:42:36.384116 6355 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } }
I20260807 19:42:36.384974 6363 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } } }
I20260807 19:42:36.385321 6363 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:36.385985 6355 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 19:42:36.394411 6375 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:42:36.394573 6375 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:42:36.397706 6279 master_runner.cc:428] Detected that this master f1aaac13b4e84290b4e10e03dbafc748 is joining an existing cluster
I20260807 19:42:36.397812 6279 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.208.61:40519
I20260807 19:42:36.417783 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.402075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59116 (local address 127.30.208.61:40519)
0807 19:42:36.402299 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:36.402305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:36.402916 (+ 611us) server_negotiation.cc:407] Connection header received
0807 19:42:36.403211 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:36.403217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:36.403292 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:36.403426 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:36.405054 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.406129 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.408003 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.408746 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.408840 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.411190 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:36.411196 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:36.411201 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:36.411238 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:36.414791 (+ 3553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.415554 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.415559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.415562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.415709 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.416123 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.416128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.416131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.416485 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:36.416631 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:36.417137 (+ 506us) server_negotiation.cc:299] Negotiation successful
0807 19:42:36.417387 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":65}
I20260807 19:42:36.418818 6382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.402145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:40519 (local address 127.0.0.1:59116)
0807 19:42:36.402785 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:36.402808 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:36.403038 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:36.403653 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:36.403658 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:36.404409 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:36.404896 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.404911 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.406309 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.406314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.407868 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.407882 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.409140 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.409148 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.409336 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.409976 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:36.409999 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:36.411034 (+ 1035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:36.414937 (+ 3903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:36.414942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:36.414946 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:36.415431 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:36.415854 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:36.415859 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:36.415863 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:36.416011 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:36.416713 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:36.416717 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:36.416890 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 19:42:36.418078 (+ 1188us) client_negotiation.cc:240] Negotiation successful
0807 19:42:36.418364 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":350,"threads_started":1}
I20260807 19:42:36.420187 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.402065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55542 (local address 127.30.208.62:33613)
0807 19:42:36.402255 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:36.402261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:36.403814 (+ 1553us) server_negotiation.cc:407] Connection header received
0807 19:42:36.403934 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:36.403939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:36.404029 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:36.404119 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:36.406631 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.407732 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.409726 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.410324 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.410449 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.413332 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:36.413338 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:36.413343 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:36.413384 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:36.416506 (+ 3122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.417213 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.417218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.417221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.417376 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.417805 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.417811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.417814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.418440 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:36.418581 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:36.419653 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0807 19:42:36.419891 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:42:36.421319 6383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.403044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:55542)
0807 19:42:36.403688 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:36.403702 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:36.403822 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:36.404138 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:36.404142 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:36.404912 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:36.406454 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.406467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.407969 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.407974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.409543 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.409556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.411260 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.411266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.411467 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.412017 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:36.412039 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:36.413093 (+ 1054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:36.416637 (+ 3544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:36.416642 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:36.416645 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:36.417070 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:36.417515 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:36.417519 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:36.417523 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:36.417671 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:36.418667 (+ 996us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:36.418672 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:36.418872 (+ 200us) client_negotiation.cc:769] Sending connection context
0807 19:42:36.420740 (+ 1868us) client_negotiation.cc:240] Negotiation successful
0807 19:42:36.420974 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":24,"thread_start_us":379,"threads_started":1}
WARNING: no policy specified for oryx/127.30.208.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.208.60@KRBTEST.COM" created.
Aug 07 19:42:36 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131754, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.62@KRBTEST.COM for oryx/127.30.208.61@KRBTEST.COM
I20260807 19:42:36.444350 6358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.427004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59124 (local address 127.30.208.61:40519)
0807 19:42:36.427178 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:36.427184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:36.427819 (+ 635us) server_negotiation.cc:407] Connection header received
0807 19:42:36.428138 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:36.428143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:36.428242 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:36.428325 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:36.429462 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.430213 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.431467 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.431961 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.432067 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.437553 (+ 5486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:36.437560 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:36.437565 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:36.437607 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:36.441079 (+ 3472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.441968 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.441974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.441978 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.442147 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.442588 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.442593 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.442597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.442977 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:36.443139 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:36.443732 (+ 593us) server_negotiation.cc:299] Negotiation successful
0807 19:42:36.443962 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:42:36.444388 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.427010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:40519 (local address 127.0.0.1:59124)
0807 19:42:36.427685 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:36.427721 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:36.427953 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:36.428403 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:36.428410 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:36.429055 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:36.429311 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.429322 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.430342 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.430347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.431336 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.431349 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.432113 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.432119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.432304 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.432854 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:36.432870 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:36.437332 (+ 4462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:36.441268 (+ 3936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:36.441284 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:36.441292 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:36.441836 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:36.442280 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:36.442285 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:36.442288 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:36.442452 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:36.443114 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:36.443121 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:36.443331 (+ 210us) client_negotiation.cc:769] Sending connection context
0807 19:42:36.443599 (+ 268us) client_negotiation.cc:240] Negotiation successful
0807 19:42:36.443990 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":298,"threads_started":1}
I20260807 19:42:36.446617 5777 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.208.61:40519: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748"
member_type: NON_VOTER
last_known_addr {
host: "127.30.208.61"
port: 40519
}
attrs {
promote: true
}
}
dest_uuid: "4b72ec6924c24c76b5bee353ab72410d"
cas_config_opid_index: -1
I20260807 19:42:36.447557 5777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: NON_VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: true } }
I20260807 19:42:36.462805 6391 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f1aaac13b4e84290b4e10e03dbafc748 (127.30.208.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: NON_VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: true } } }
I20260807 19:42:36.463846 6327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:36.467252 6391 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.208.61:40519
I20260807 19:42:36.467445 6327 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 FOLLOWER]: Refusing update from remote peer 4b72ec6924c24c76b5bee353ab72410d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 19:42:36.467890 6390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: NON_VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: true } } }
I20260807 19:42:36.468227 6390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:36.468242 6279 master.cc:537] Master@127.30.208.61:40519 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.208.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.60.keytab.
Entry for principal oryx/127.30.208.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.60.keytab.
I20260807 19:42:36.470044 6390 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: NON_VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:36.471434 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.60:41979
--webserver_interface=127.30.208.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.60.keytab
--principal=oryx/127.30.208.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:33613,127.30.208.61:40519,127.30.208.60:41979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:36.474301 5762 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260807 19:42:36.481535 6279 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:36.482057 6279 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:36.482362 6279 tablet_replica.cc:330] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: stopping tablet replica
W20260807 19:42:36.483609 5762 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d -> Peer f1aaac13b4e84290b4e10e03dbafc748 (127.30.208.61:40519): Couldn't send request to peer f1aaac13b4e84290b4e10e03dbafc748. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:42:36.488946 6279 master.cc:552] Master@127.30.208.61:40519 shutdown complete.
I20260807 19:42:36.489111 6279 master_runner.cc:305] Clearing existing system tablet
I20260807 19:42:36.489640 6279 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:36.489778 6279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:36.492859 6279 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 19:42:36.494740 6397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:36.495208 6279 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:36.495604 6279 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 19:42:36.495698 6279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/wal
uuid: "f1aaac13b4e84290b4e10e03dbafc748"
format_stamp: "Formatted at 2026-08-07 19:42:36 on dist-test-slave-vq2q"
server_key: "20d6c7c82dd78ccc08beefe9cbfe9d59"
server_key_iv: "2cc5fabef4d8f851437c648607462679"
server_key_version: "encryptionkey@0"
I20260807 19:42:36.500979 6279 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:36.508068 6279 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 19:42:36.508260 6279 log.cc:1196] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 19:42:36.508700 6279 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: Deleting consensus metadata
I20260807 19:42:36.508972 6279 master_runner.cc:315] Copying system tablet from 127.30.208.62:33613
I20260807 19:42:36.511493 6279 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: tablet copy: Beginning tablet copy session from remote peer at address 127.30.208.62:33613
I20260807 19:42:36.527863 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.512397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55546 (local address 127.30.208.62:33613)
0807 19:42:36.512575 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:36.512581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:36.513661 (+ 1080us) server_negotiation.cc:407] Connection header received
0807 19:42:36.513954 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:36.513961 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:36.514050 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:36.514156 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:36.515183 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.516334 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.518068 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.518741 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.518882 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.521223 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:36.521231 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:36.521236 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:36.521281 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:36.524696 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.525587 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.525593 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.525596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.525773 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.526220 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.526225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.526229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.526616 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:36.526767 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:36.527248 (+ 481us) server_negotiation.cc:299] Negotiation successful
0807 19:42:36.527491 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:42:36.527969 6403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.512270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:55546)
0807 19:42:36.513550 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:36.513573 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:36.513782 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:36.514208 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:36.514213 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:36.514756 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:36.515020 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.515033 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.516498 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.516503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.517902 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.517918 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.518917 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.518923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.519107 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.519853 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:36.519877 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:36.520961 (+ 1084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:36.524866 (+ 3905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:36.524872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:36.524876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:36.525465 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:36.525897 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:36.525902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:36.525905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:36.526100 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:36.526759 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:36.526764 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:36.526955 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 19:42:36.527114 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 19:42:36.527442 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1142,"thread_start_us":1046,"threads_started":1}
I20260807 19:42:36.529361 5798 tablet_copy_service.cc:140] P 4b72ec6924c24c76b5bee353ab72410d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f1aaac13b4e84290b4e10e03dbafc748 ({username='oryx', principal='oryx/127.30.208.61@KRBTEST.COM'} at 127.0.0.1:55546)
I20260807 19:42:36.529650 5798 tablet_copy_service.cc:161] P 4b72ec6924c24c76b5bee353ab72410d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f1aaac13b4e84290b4e10e03dbafc748 at {username='oryx', principal='oryx/127.30.208.61@KRBTEST.COM'} at 127.0.0.1:55546: session id = f1aaac13b4e84290b4e10e03dbafc748-00000000000000000000000000000000
I20260807 19:42:36.534196 5798 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:42:36.536649 6279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 19:42:36.543812 6279 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: tablet copy: Starting download of 0 data blocks...
I20260807 19:42:36.544293 6279 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: tablet copy: Starting download of 1 WAL segments...
I20260807 19:42:36.547256 6279 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:42:36.551862 5798 tablet_copy_service.cc:342] P 4b72ec6924c24c76b5bee353ab72410d: Request end of tablet copy session f1aaac13b4e84290b4e10e03dbafc748-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.208.61@KRBTEST.COM'} at 127.0.0.1:55546
I20260807 19:42:36.552107 5798 tablet_copy_service.cc:433] P 4b72ec6924c24c76b5bee353ab72410d: ending tablet copy session f1aaac13b4e84290b4e10e03dbafc748-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f1aaac13b4e84290b4e10e03dbafc748
I20260807 19:42:36.564642 6279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:36.567368 6405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:36.577543 6406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:36.577955 6408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:36.578310 6279 server_base.cc:1034] running on GCE node
I20260807 19:42:36.578709 6279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:36.579420 6279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:36.580705 6279 hybrid_clock.cc:648] HybridClock initialized: now 1786131756580640 us; error 34 us; skew 500 ppm
I20260807 19:42:36.580938 6279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:36 dist-test-slave-vq2q krb5kdc[5729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131756, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:36.583097 6279 init.cc:377] Logged in from keytab as oryx/127.30.208.61@KRBTEST.COM (short username oryx)
I20260807 19:42:36.584321 6279 webserver.cc:460] Webserver started at http://127.30.208.61:42181/ using document root <none> and password file <none>
I20260807 19:42:36.584718 6279 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:36.584851 6279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:36.588019 6279 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 19:42:36.590013 6415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:36.590433 6279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260807 19:42:36.590580 6279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/wal
uuid: "f1aaac13b4e84290b4e10e03dbafc748"
format_stamp: "Formatted at 2026-08-07 19:42:36 on dist-test-slave-vq2q"
server_key: "20d6c7c82dd78ccc08beefe9cbfe9d59"
server_key_iv: "2cc5fabef4d8f851437c648607462679"
server_key_version: "encryptionkey@0"
I20260807 19:42:36.590750 6279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:36.622481 6279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:36.626601 6279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:36.649838 6279 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.61:40519
I20260807 19:42:36.649838 6471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.61:40519 every 8 connection(s)
I20260807 19:42:36.651089 6279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-1/data/info.pb
I20260807 19:42:36.659989 6472 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:42:36.663883 6472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: Bootstrap starting.
I20260807 19:42:36.681847 6472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:42:36.682410 6472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: Bootstrap complete.
I20260807 19:42:36.683495 6472 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: NON_VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: true } }
I20260807 19:42:36.683760 6472 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f1aaac13b4e84290b4e10e03dbafc748, State: Initialized, Role: LEARNER
I20260807 19:42:36.684041 6472 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: NON_VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: true } }
I20260807 19:42:36.684932 6475 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: NON_VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: true } } }
I20260807 19:42:36.685194 6472 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:36.685343 6475 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: This master's current role is: LEARNER
I20260807 19:42:36.696529 6488 catalog_manager.cc:1259] Loaded cluster ID: 48ade388e5c844bf927e36d1c2fe5952
I20260807 19:42:36.696759 6488 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: loading cluster ID for follower catalog manager: success
I20260807 19:42:36.700556 6488 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: acquiring CA information for follower catalog manager: success
I20260807 19:42:36.703401 6488 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:42:36.831173 6274 heartbeater.cc:503] Master 127.30.208.62:33613 was elected leader, sending a full tablet report...
W20260807 19:42:36.859387 6395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:36.859694 6395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:36.859831 6395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:36.867115 6395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:36.867244 6395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:36.867328 6395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:36.867408 6395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:36.867470 6395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:36.877611 6395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:33613,127.30.208.61:40519,127.30.208.60:41979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.60.keytab
--principal=oryx/127.30.208.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.60:41979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.208.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:36.879317 6395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:36.880602 6395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:36.892493 6493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:36.892643 6494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:36.892541 6497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:36.893603 6395 server_base.cc:1034] running on GCE node
I20260807 19:42:36.894466 6395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:36.895987 6395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:36.897219 6395 hybrid_clock.cc:648] HybridClock initialized: now 1786131756897137 us; error 48 us; skew 500 ppm
I20260807 19:42:36.897547 6395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:36 dist-test-slave-vq2q krb5kdc[5729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131756, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:36.900655 6395 init.cc:377] Logged in from keytab as oryx/127.30.208.60@KRBTEST.COM (short username oryx)
I20260807 19:42:36.902195 6395 webserver.cc:460] Webserver started at http://127.30.208.60:34141/ using document root <none> and password file <none>
I20260807 19:42:36.902840 6395 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:36.902974 6395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:36.903263 6395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:36.906365 6395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/data/instance:
uuid: "61adb8b777d7419196442d1345e0dd55"
format_stamp: "Formatted at 2026-08-07 19:42:36 on dist-test-slave-vq2q"
server_key: "3e584e1bdf2107ba10d57a26bfd1e8a6"
server_key_iv: "55d5a298257d76b1ec5b20afd7ba1230"
server_key_version: "encryptionkey@0"
I20260807 19:42:36.907107 6395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/wal/instance:
uuid: "61adb8b777d7419196442d1345e0dd55"
format_stamp: "Formatted at 2026-08-07 19:42:36 on dist-test-slave-vq2q"
server_key: "3e584e1bdf2107ba10d57a26bfd1e8a6"
server_key_iv: "55d5a298257d76b1ec5b20afd7ba1230"
server_key_version: "encryptionkey@0"
I20260807 19:42:36.912595 6395 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:42:36.916036 6504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:36.917526 6395 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:36.917721 6395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/wal
uuid: "61adb8b777d7419196442d1345e0dd55"
format_stamp: "Formatted at 2026-08-07 19:42:36 on dist-test-slave-vq2q"
server_key: "3e584e1bdf2107ba10d57a26bfd1e8a6"
server_key_iv: "55d5a298257d76b1ec5b20afd7ba1230"
server_key_version: "encryptionkey@0"
I20260807 19:42:36.917922 6395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:36.940327 6395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:36.946141 6395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:36.946424 6395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:36.962582 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.954366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:40519 (local address 127.0.0.1:59134)
0807 19:42:36.955378 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:36.955398 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:36.955558 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:36.956021 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:36.956027 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:36.956066 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:36.956311 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.956325 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.957894 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.957900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.960010 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.960032 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.961701 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.961709 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.961914 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.961989 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 19:42:36.962094 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 19:42:36.962248 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"thread_start_us":754,"threads_started":1}
I20260807 19:42:36.962569 6526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.954530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59134 (local address 127.30.208.61:40519)
0807 19:42:36.955190 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:36.955197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:36.955556 (+ 359us) server_negotiation.cc:407] Connection header received
0807 19:42:36.955687 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:36.955691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:36.955798 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:36.955925 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:36.956505 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.957731 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.960247 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.961496 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.961644 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.961908 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:36.962141 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 19:42:36.962275 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":399,"threads_started":1}
I20260807 19:42:36.963763 6444 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260807 19:42:36.964366 6475 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b72ec6924c24c76b5bee353ab72410d. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: NON_VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: true } } }
I20260807 19:42:36.964641 6475 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: This master's current role is: LEARNER
I20260807 19:42:36.966394 6395 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.60:41979
I20260807 19:42:36.966404 6561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.60:41979 every 8 connection(s)
I20260807 19:42:36.967727 6395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/data/info.pb
I20260807 19:42:36.971411 6562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:36.971875 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 6395
I20260807 19:42:36.980080 6562 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:42:36 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.208.60@KRBTEST.COM
Aug 07 19:42:36 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131756, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.60@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
I20260807 19:42:36.994151 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.973906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35354 (local address 127.30.208.60:41979)
0807 19:42:36.974671 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:36.974683 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:36.974728 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:42:36.976226 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:36.976248 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:36.976624 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:36.976998 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:36.978553 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.979807 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.981142 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.981866 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.982039 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.986907 (+ 4868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:36.986915 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:36.986926 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:36.986966 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:36.990012 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.990766 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.990773 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.990779 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.990956 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.991321 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.991325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.991327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.991930 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:36.992193 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:36.992665 (+ 472us) server_negotiation.cc:299] Negotiation successful
0807 19:42:36.993049 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":337,"threads_started":1}
I20260807 19:42:36.997071 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.982810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55548 (local address 127.30.208.62:33613)
0807 19:42:36.983017 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:36.983023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:36.983424 (+ 401us) server_negotiation.cc:407] Connection header received
0807 19:42:36.983596 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:36.983601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:36.983667 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:36.983749 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:36.984990 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.986014 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.987449 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.987833 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.987955 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.991315 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:36.991322 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:36.991327 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:36.991367 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:36.994160 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.994963 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.994969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.994972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.995151 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:36.995568 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:36.995573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:36.995577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:36.995853 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:36.995998 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:36.996477 (+ 479us) server_negotiation.cc:299] Negotiation successful
0807 19:42:36.996715 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:42:36.997123 6567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.982626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:55548)
0807 19:42:36.983284 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:36.983320 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:36.983475 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:36.983825 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:36.983832 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:36.984552 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:36.984835 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.984846 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.986156 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.986163 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.987304 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.987317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:36.988019 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:36.988025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:36.988216 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:36.988766 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:36.988785 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:36.991146 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:36.994305 (+ 3159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:36.994320 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:36.994326 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:36.994852 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:36.995258 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:36.995261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:36.995264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:36.995464 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:36.996014 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:36.996021 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:36.996180 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 19:42:36.996361 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 19:42:36.996739 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":307,"threads_started":1}
I20260807 19:42:36.998319 6562 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:42:37 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131756, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.60@KRBTEST.COM for oryx/127.30.208.61@KRBTEST.COM
I20260807 19:42:37.010461 6567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.999013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:40519 (local address 127.0.0.1:59150)
0807 19:42:36.999188 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:36.999206 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:36.999292 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:36.999516 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:36.999519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:36.999812 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:36.999965 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:36.999984 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.000944 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.000948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.001916 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.001927 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.002582 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.002586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.002721 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.003299 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:37.003316 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:37.005381 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:37.008311 (+ 2930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.008315 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.008318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.008701 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.009053 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.009056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.009058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.009261 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.009716 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:37.009718 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:37.009847 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 19:42:37.009945 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 19:42:37.010219 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 19:42:37.010475 6526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:36.999127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59150 (local address 127.30.208.61:40519)
0807 19:42:36.999283 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:36.999287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:36.999303 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:42:36.999363 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:36.999367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:36.999424 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:36.999497 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:37.000115 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.000827 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.002062 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.002469 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.002566 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.005563 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:37.005568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:37.005572 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:37.005608 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:37.008171 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.008836 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.008841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.008843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.008974 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.009355 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.009358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.009360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.009635 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:37.009745 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.010030 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.010257 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 19:42:37.011636 6562 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:42:37 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131756, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.60@KRBTEST.COM for oryx/127.30.208.60@KRBTEST.COM
I20260807 19:42:37.023756 6567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.012322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:41979 (local address 127.0.0.1:35362)
0807 19:42:37.012495 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:37.012512 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:37.012618 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:37.012942 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:37.012946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:37.013387 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:37.013535 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.013544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.014477 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.014482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.015293 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.015305 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.015960 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.015966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.016140 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.016689 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:37.016705 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:37.018724 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:37.021507 (+ 2783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.021512 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.021516 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.021854 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.022176 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.022181 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.022185 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.022333 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.022821 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:37.022826 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:37.022960 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 19:42:37.023071 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 19:42:37.023330 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 19:42:37.023824 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.012502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35362 (local address 127.30.208.60:41979)
0807 19:42:37.012656 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:37.012662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:37.012678 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:42:37.012767 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:37.012771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:37.012825 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:37.012905 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:37.013670 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.014357 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.015425 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.015812 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.015918 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.018887 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:37.018892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:37.018895 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:37.018922 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:37.021390 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.021958 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.021962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.021964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.022069 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.022428 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.022433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.022436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.022706 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:37.022841 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.023165 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.023493 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":110,"server-negotiator.queue_time_us":52}
I20260807 19:42:37.028570 6562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: Bootstrap starting.
I20260807 19:42:37.031840 6562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:37.032904 6562 log.cc:824] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:37.035607 6562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: No bootstrap required, opened a new log
I20260807 19:42:37.042722 6562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } }
I20260807 19:42:37.043062 6562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:37.043181 6562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61adb8b777d7419196442d1345e0dd55, State: Initialized, Role: FOLLOWER
I20260807 19:42:37.043740 6562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } }
I20260807 19:42:37.044591 6569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } } }
I20260807 19:42:37.044921 6569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:37.045645 6562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 19:42:37.054049 6581 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:42:37.054203 6581 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:42:37.059185 6395 master_runner.cc:428] Detected that this master 61adb8b777d7419196442d1345e0dd55 is joining an existing cluster
I20260807 19:42:37.059297 6395 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.208.60:41979
I20260807 19:42:37.080619 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.064080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55558 (local address 127.30.208.62:33613)
0807 19:42:37.064272 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:37.064276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:37.064297 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:42:37.064476 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:37.064481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:37.064568 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:37.064646 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:37.066646 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.067828 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.069982 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.070631 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.070738 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.073651 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:37.073659 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:37.073663 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:37.073709 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:37.077119 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.078013 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.078018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.078022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.078211 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.078723 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.078729 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.078731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.079110 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:37.079257 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.079944 (+ 687us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.080217 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:42:37.081080 6526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.063446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59158 (local address 127.30.208.61:40519)
0807 19:42:37.063925 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:37.063931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:37.065130 (+ 1199us) server_negotiation.cc:407] Connection header received
0807 19:42:37.065434 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:37.065440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:37.065520 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:37.065587 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:37.066981 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.068006 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.070728 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.071374 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.071470 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.074050 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:37.074057 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:37.074061 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:37.074101 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:37.077961 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.078740 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.078745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.078748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.078902 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.079388 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.079393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.079396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.079758 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:37.079882 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.080497 (+ 615us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.080751 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339}
I20260807 19:42:37.081738 6587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.063376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:55558)
0807 19:42:37.064147 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:37.064171 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:37.064307 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:37.064910 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:37.064915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:37.066064 (+ 1149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:37.066444 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.066456 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.068008 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.068013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.069824 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.069837 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.071042 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.071049 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.071250 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.071958 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:37.071982 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:37.073314 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:37.077328 (+ 4014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.077333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.077337 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.077853 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.078359 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.078363 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.078367 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.078584 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.079335 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:37.079340 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:37.079582 (+ 242us) client_negotiation.cc:769] Sending connection context
0807 19:42:37.080901 (+ 1319us) client_negotiation.cc:240] Negotiation successful
0807 19:42:37.081269 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":2,"thread_start_us":921,"threads_started":1}
I20260807 19:42:37.082357 6588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.063487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:40519 (local address 127.0.0.1:59158)
0807 19:42:37.065030 (+ 1543us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:37.065051 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:37.065265 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:37.065828 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:37.065833 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:37.066465 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:37.066816 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.066829 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.068202 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.068207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.070564 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.070579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.071787 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.071795 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.071964 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.072552 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:37.072578 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:37.073813 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:37.078121 (+ 4308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.078126 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.078129 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.078584 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.079063 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.079068 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.079072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.079255 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.079967 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:37.079972 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:37.080203 (+ 231us) client_negotiation.cc:769] Sending connection context
0807 19:42:37.081335 (+ 1132us) client_negotiation.cc:240] Negotiation successful
0807 19:42:37.081698 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1419,"mutex_wait_us":743,"thread_start_us":1217,"threads_started":1}
I20260807 19:42:37.087344 6589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.065130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:41979 (local address 127.0.0.1:35376)
0807 19:42:37.069434 (+ 4304us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:37.069452 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:37.069574 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:37.070021 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:37.070026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:37.070754 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:37.071125 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.071138 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.072719 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.072727 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.074419 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.074434 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.076103 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.076110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.076315 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.076897 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:37.076935 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:37.078142 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:37.084304 (+ 6162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.084309 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.084313 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.084829 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.085281 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.085286 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.085290 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.085434 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.086191 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:37.086196 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:37.086362 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 19:42:37.086573 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 19:42:37.086848 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4143,"thread_start_us":3975,"threads_started":1}
I20260807 19:42:37.087443 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.065587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35376 (local address 127.30.208.60:41979)
0807 19:42:37.065773 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:37.065779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:37.069542 (+ 3763us) server_negotiation.cc:407] Connection header received
0807 19:42:37.069687 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:37.069692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:37.069774 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:37.069875 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:37.071280 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.072447 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.074948 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.075671 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.075808 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.078337 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:37.078343 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:37.078347 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:37.078383 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:37.084159 (+ 5776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.084963 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.084972 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.084976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.085149 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.085550 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.085555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.085558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.085995 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:37.086189 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.086860 (+ 671us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.087087 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 19:42:37 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131754, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.62@KRBTEST.COM for oryx/127.30.208.60@KRBTEST.COM
I20260807 19:42:37.099812 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 5845
I20260807 19:42:37.101439 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.087838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:41979 (local address 127.0.0.1:35378)
0807 19:42:37.088000 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:37.088017 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:37.088119 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:37.088483 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:37.088488 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:37.088919 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:37.089120 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.089134 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.090241 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.090247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.091311 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.091324 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.092134 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.092140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.092324 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.092858 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:37.092875 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:37.094993 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:37.098476 (+ 3483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.098482 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.098486 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.099016 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.099571 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.099576 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.099580 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.099757 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.100392 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:37.100397 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:37.100589 (+ 192us) client_negotiation.cc:769] Sending connection context
0807 19:42:37.100744 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 19:42:37.101040 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 19:42:37.102036 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.087987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35378 (local address 127.30.208.60:41979)
0807 19:42:37.088135 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:37.088141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:37.088157 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:42:37.088249 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:37.088259 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:37.088353 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:37.088432 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:37.089276 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.090115 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.091447 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.091983 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.092112 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.095172 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:37.095176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:37.095179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:37.095207 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:37.098325 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.099131 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.099137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.099140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.099301 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.099891 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.099896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.099899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.100258 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:37.100410 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.101373 (+ 963us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.101641 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 19:42:37.103494 5777 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.208.60:41979: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "61adb8b777d7419196442d1345e0dd55"
member_type: NON_VOTER
last_known_addr {
host: "127.30.208.60"
port: 41979
}
attrs {
promote: true
}
}
dest_uuid: "4b72ec6924c24c76b5bee353ab72410d"
cas_config_opid_index: 5
I20260807 19:42:37.104565 5777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: NON_VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: true } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } }
I20260807 19:42:37.106798 6591 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 61adb8b777d7419196442d1345e0dd55 (127.30.208.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: NON_VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: true } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } } }
I20260807 19:42:37.108202 6535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:37.111135 6591 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.208.60:41979
I20260807 19:42:37.111877 6535 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 1 FOLLOWER]: Refusing update from remote peer 4b72ec6924c24c76b5bee353ab72410d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 19:42:37.112704 6444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 LEARNER]: Refusing update from remote peer 4b72ec6924c24c76b5bee353ab72410d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 19:42:37.114167 6395 master.cc:537] Master@127.30.208.60:41979 shutting down...
I20260807 19:42:37.114205 6593 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: NON_VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260807 19:42:37.115705 6593 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [LEADER]: Connected to new peer: Peer: permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:42:37.122696 5762 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d -> Peer 61adb8b777d7419196442d1345e0dd55 (127.30.208.60:41979): Couldn't send request to peer 61adb8b777d7419196442d1345e0dd55. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 19:42:37.122453 6444 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 61adb8b777d7419196442d1345e0dd55 (127.30.208.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: NON_VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: true } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } } }
Aug 07 19:42:37 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.208.1@KRBTEST.COM
I20260807 19:42:37.125983 6390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: NON_VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: true } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } } }
I20260807 19:42:37.126312 6390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:37.133776 6590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.101163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44482 (local address 127.30.208.1:35269)
0807 19:42:37.110998 (+ 9835us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:37.111008 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:37.111039 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:42:37.111115 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:37.111124 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:37.111410 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:37.113364 (+ 1954us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:37.114976 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.116372 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.118552 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.119320 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.120149 (+ 829us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.124826 (+ 4677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:37.124848 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:37.124863 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:37.124909 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:37.128916 (+ 4007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.129954 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.129966 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.129970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.130161 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.130678 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.130683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.130687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.131180 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:37.131506 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.132960 (+ 1454us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.133304 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9659,"thread_start_us":9436,"threads_started":1}
I20260807 19:42:37.134613 6395 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:37.135097 6395 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:37.135246 5978 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:37.135295 6395 tablet_replica.cc:330] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: stopping tablet replica
I20260807 19:42:37.141134 6475 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: NON_VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: true } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } } }
I20260807 19:42:37.141510 6475 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: This master's current role is: LEARNER
I20260807 19:42:37.142023 6593 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d: attempting to promote NON_VOTER f1aaac13b4e84290b4e10e03dbafc748 to VOTER
I20260807 19:42:37.143004 6395 master.cc:552] Master@127.30.208.60:41979 shutdown complete.
I20260807 19:42:37.143079 6395 master_runner.cc:305] Clearing existing system tablet
I20260807 19:42:37.142997 6593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } }
I20260807 19:42:37.143481 6395 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:37.143594 6395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:37.145359 6444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 LEARNER]: Refusing update from remote peer 4b72ec6924c24c76b5bee353ab72410d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 19:42:37.145397 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.144552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:41979 (local address 127.0.0.1:35390)
0807 19:42:37.144710 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:37.145094 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:41979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 19:42:37.146766 6591 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 19:42:37.147176 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.146105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:41979 (local address 127.0.0.1:35404)
0807 19:42:37.146848 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:37.147019 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:41979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":607}
W20260807 19:42:37.147410 5762 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d -> Peer 61adb8b777d7419196442d1345e0dd55 (127.30.208.60:41979): Couldn't send request to peer 61adb8b777d7419196442d1345e0dd55. Status: Network error: Client connection negotiation failed: client connection to 127.30.208.60:41979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:42:37.152241 6592 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f1aaac13b4e84290b4e10e03dbafc748 (127.30.208.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } } }
I20260807 19:42:37.156759 6395 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.004s sys 0.000s
I20260807 19:42:37.157610 6444 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f1aaac13b4e84290b4e10e03dbafc748 (127.30.208.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } } }
I20260807 19:42:37.161742 6475 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } } }
I20260807 19:42:37.162020 6475 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:37.162909 6593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } } }
I20260807 19:42:37.163187 6593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:37.162950 6390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } } }
I20260807 19:42:37.163499 6390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:37.169480 6599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:37.169947 6395 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:37.170212 6395 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.002s sys 0.004s
I20260807 19:42:37.170323 6395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/wal
uuid: "61adb8b777d7419196442d1345e0dd55"
format_stamp: "Formatted at 2026-08-07 19:42:36 on dist-test-slave-vq2q"
server_key: "3e584e1bdf2107ba10d57a26bfd1e8a6"
server_key_iv: "55d5a298257d76b1ec5b20afd7ba1230"
server_key_version: "encryptionkey@0"
I20260807 19:42:37.175740 6395 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:37.182818 6395 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 19:42:37.182989 6395 log.cc:1196] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 19:42:37.183468 6395 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: Deleting consensus metadata
I20260807 19:42:37.183743 6395 master_runner.cc:315] Copying system tablet from 127.30.208.62:33613
I20260807 19:42:37.186465 6395 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: tablet copy: Beginning tablet copy session from remote peer at address 127.30.208.62:33613
I20260807 19:42:37.217542 6359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.187349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55568 (local address 127.30.208.62:33613)
0807 19:42:37.187577 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:37.187583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:37.201889 (+ 14306us) server_negotiation.cc:407] Connection header received
0807 19:42:37.202085 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:37.202091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:37.202184 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:37.202299 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:37.203402 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.204539 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.206517 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.207186 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.207325 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.210390 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:37.210397 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:37.210402 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:37.210615 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:37.214322 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.215191 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.215198 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.215202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.215366 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.215798 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.215803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.215806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.216181 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:37.216348 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.216900 (+ 552us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.217148 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 19:42:37.217577 6605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.187264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:55568)
0807 19:42:37.201694 (+ 14430us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:37.201728 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:37.201943 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:37.202346 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:37.202352 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:37.202954 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:37.203219 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.203232 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.204693 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.204698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.206314 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.206330 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.207378 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.207385 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.207587 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.208321 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:37.208353 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:37.210154 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:37.214502 (+ 4348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.214508 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.214513 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.215046 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.215480 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.215485 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.215489 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.215675 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.216322 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:37.216327 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:37.216528 (+ 201us) client_negotiation.cc:769] Sending connection context
0807 19:42:37.216717 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 19:42:37.217071 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14255,"thread_start_us":14122,"threads_started":1}
I20260807 19:42:37.218657 5798 tablet_copy_service.cc:140] P 4b72ec6924c24c76b5bee353ab72410d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 61adb8b777d7419196442d1345e0dd55 ({username='oryx', principal='oryx/127.30.208.60@KRBTEST.COM'} at 127.0.0.1:55568)
I20260807 19:42:37.218835 5798 tablet_copy_service.cc:161] P 4b72ec6924c24c76b5bee353ab72410d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 61adb8b777d7419196442d1345e0dd55 at {username='oryx', principal='oryx/127.30.208.60@KRBTEST.COM'} at 127.0.0.1:55568: session id = 61adb8b777d7419196442d1345e0dd55-00000000000000000000000000000000
I20260807 19:42:37.222582 5798 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 19:42:37.224519 6395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 19:42:37.231654 6395 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: tablet copy: Starting download of 0 data blocks...
I20260807 19:42:37.232000 6395 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: tablet copy: Starting download of 1 WAL segments...
I20260807 19:42:37.234817 6395 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 19:42:37.239430 5798 tablet_copy_service.cc:342] P 4b72ec6924c24c76b5bee353ab72410d: Request end of tablet copy session 61adb8b777d7419196442d1345e0dd55-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.208.60@KRBTEST.COM'} at 127.0.0.1:55568
I20260807 19:42:37.239580 5798 tablet_copy_service.cc:433] P 4b72ec6924c24c76b5bee353ab72410d: ending tablet copy session 61adb8b777d7419196442d1345e0dd55-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 61adb8b777d7419196442d1345e0dd55
I20260807 19:42:37.252113 6395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:37.255267 6606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:37.263633 6609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:37.264448 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 5845
W20260807 19:42:37.273604 6607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:37.276152 6395 server_base.cc:1034] running on GCE node
I20260807 19:42:37.276487 6395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:37.277130 6395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:37.280445 6395 hybrid_clock.cc:648] HybridClock initialized: now 1786131757280406 us; error 41 us; skew 500 ppm
I20260807 19:42:37.280674 6395 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:37 dist-test-slave-vq2q krb5kdc[5729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131757, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:37.282570 6395 init.cc:377] Logged in from keytab as oryx/127.30.208.60@KRBTEST.COM (short username oryx)
I20260807 19:42:37.285888 6395 webserver.cc:460] Webserver started at http://127.30.208.60:42513/ using document root <none> and password file <none>
I20260807 19:42:37.286226 6395 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:37.286327 6395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:37.288738 6395 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 19:42:37.293735 6616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:37.294227 6395 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:42:37.294350 6395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/wal
uuid: "61adb8b777d7419196442d1345e0dd55"
format_stamp: "Formatted at 2026-08-07 19:42:36 on dist-test-slave-vq2q"
server_key: "3e584e1bdf2107ba10d57a26bfd1e8a6"
server_key_iv: "55d5a298257d76b1ec5b20afd7ba1230"
server_key_version: "encryptionkey@0"
I20260807 19:42:37.294553 6395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:37.301944 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:35269
--local_ip_for_outbound_sockets=127.30.208.1
--tserver_master_addrs=127.30.208.62:33613
--webserver_port=39937
--webserver_interface=127.30.208.1
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.1.keytab
--principal=oryx/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.208.62:33613,127.30.208.61:40519
--tserver_master_addrs=127.30.208.62:33613,127.30.208.61:40519,127.30.208.60:41979 with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:42:37.305233 6395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:37.308562 6395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:37.344249 6395 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.60:41979
I20260807 19:42:37.344269 6673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.60:41979 every 8 connection(s)
I20260807 19:42:37.345130 6395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/master-2/data/info.pb
I20260807 19:42:37.350633 6674 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:42:37.354856 6674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: Bootstrap starting.
I20260807 19:42:37.369764 6674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:42:37.370203 6674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: Bootstrap complete.
I20260807 19:42:37.371099 6674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } }
I20260807 19:42:37.371337 6674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 61adb8b777d7419196442d1345e0dd55, State: Initialized, Role: LEARNER
I20260807 19:42:37.371577 6674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } }
I20260807 19:42:37.372342 6677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } } }
I20260807 19:42:37.372684 6674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:37.372889 6677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [sys.catalog]: This master's current role is: LEARNER
I20260807 19:42:37.384121 6691 catalog_manager.cc:1259] Loaded cluster ID: 48ade388e5c844bf927e36d1c2fe5952
I20260807 19:42:37.384256 6691 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: loading cluster ID for follower catalog manager: success
I20260807 19:42:37.387840 6691 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: acquiring CA information for follower catalog manager: success
I20260807 19:42:37.390301 6691 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:42:37.640595 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.634120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35410 (local address 127.30.208.60:41979)
0807 19:42:37.634861 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:37.634866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:37.634882 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:42:37.634944 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:37.634948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:37.635019 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:37.635115 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:37.635618 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.636580 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.638838 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.639651 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.639763 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.639963 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.640222 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.640327 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":455,"threads_started":1}
I20260807 19:42:37.640676 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.633999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:41979 (local address 127.0.0.1:35410)
0807 19:42:37.634190 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:37.634212 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:37.634356 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:37.635201 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:37.635207 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:37.635254 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:37.635476 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.635490 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.636750 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.636755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.638638 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.638659 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.639840 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.639847 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.640062 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.640100 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:42:37.640212 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 19:42:37.640365 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 19:42:37.641734 6647 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 19:42:37.642333 6677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b72ec6924c24c76b5bee353ab72410d. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } } }
I20260807 19:42:37.642614 6677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [sys.catalog]: This master's current role is: LEARNER
W20260807 19:42:37.667568 6624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:37.667855 6624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:37.667991 6624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:37.675472 6624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:37.675601 6624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:37.675762 6624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:42:37.685906 6624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.1.keytab
--principal=oryx/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:35269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=39937
--tserver_master_addrs=127.30.208.62:33613,127.30.208.61:40519,127.30.208.60:41979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:37.687247 6624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:37.688704 6624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:37.702134 6697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:37.702190 6700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:37.702234 6698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:37.703150 6624 server_base.cc:1034] running on GCE node
I20260807 19:42:37.703831 6624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:37.704797 6624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:37.706064 6624 hybrid_clock.cc:648] HybridClock initialized: now 1786131757706016 us; error 36 us; skew 500 ppm
I20260807 19:42:37.706429 6624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:37 dist-test-slave-vq2q krb5kdc[5729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131757, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:37.709645 6624 init.cc:377] Logged in from keytab as oryx/127.30.208.1@KRBTEST.COM (short username oryx)
I20260807 19:42:37.711270 6624 webserver.cc:460] Webserver started at http://127.30.208.1:39937/ using document root <none> and password file <none>
I20260807 19:42:37.711959 6624 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:37.712078 6624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:37.717458 6624 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:37.720280 6708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:37.721720 6624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:37.721911 6624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee"
format_stamp: "Formatted at 2026-08-07 19:42:34 on dist-test-slave-vq2q"
server_key: "fedbbb89f8d40355777362095057f92a"
server_key_iv: "d9f4ab4706f32762d97963a92cbc9be0"
server_key_version: "encryptionkey@0"
I20260807 19:42:37.722890 6624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:37.770082 6624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:37.774325 6624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:37.774621 6624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:37.775861 6624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:37.777655 6624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:37.777777 6624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:37.777899 6624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:37.777976 6624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:37.817127 6624 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:35269
I20260807 19:42:37.817230 6835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:35269 every 8 connection(s)
I20260807 19:42:37.818627 6624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:37.823010 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 6624
I20260807 19:42:37.823344 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 5991
Aug 07 19:42:37 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131757, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
I20260807 19:42:37.860174 6841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.821593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:57299 (local address 127.30.208.62:33613)
0807 19:42:37.842253 (+ 20660us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:37.842260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:37.842285 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:42:37.842355 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:37.842361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:37.842453 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:37.842584 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:37.844787 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.845625 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.847314 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.847712 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.847831 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.852503 (+ 4672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:37.852509 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:37.852514 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:37.852556 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:37.855961 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.856823 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.856828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.856832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.856994 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.857511 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.857516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.857520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.857860 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:37.858003 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.859556 (+ 1553us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.859787 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20470,"thread_start_us":20310,"threads_started":1}
Aug 07 19:42:37 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131757, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.60@KRBTEST.COM
I20260807 19:42:37.861495 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.822165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.1:57299)
0807 19:42:37.837945 (+ 15780us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:37.838026 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:37.840963 (+ 2937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:37.842938 (+ 1975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:37.842951 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:37.843869 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:37.844577 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.844599 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.845812 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.845817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.847161 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.847175 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.847954 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.847962 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.848214 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.848983 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:37.849012 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:37.852222 (+ 3210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:37.856085 (+ 3863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.856094 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.856111 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.856700 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.857095 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.857099 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.857103 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.857402 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.858621 (+ 1219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:37.858629 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:37.859097 (+ 468us) client_negotiation.cc:769] Sending connection context
0807 19:42:37.859438 (+ 341us) client_negotiation.cc:240] Negotiation successful
0807 19:42:37.860003 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15383,"thread_start_us":15118,"threads_started":1}
Aug 07 19:42:37 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.208.2@KRBTEST.COM
I20260807 19:42:37.863639 6837 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33613
I20260807 19:42:37.863994 6837 heartbeater.cc:464] Registering TS with master...
Aug 07 19:42:37 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131757, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.61@KRBTEST.COM
I20260807 19:42:37.864928 6837 heartbeater.cc:511] Master 127.30.208.62:33613 requested a full tablet report, sending...
I20260807 19:42:37.867261 5777 ts_manager.cc:194] Re-registered known tserver with Master: 3a6ac8ef1cb744cbbf4d3916bf4d44ee (127.30.208.1:35269)
I20260807 19:42:37.868362 5777 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:57299
I20260807 19:42:37.868712 6692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.851069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:48449 (local address 127.30.208.60:41979)
0807 19:42:37.851829 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:37.851836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:37.852614 (+ 778us) server_negotiation.cc:407] Connection header received
0807 19:42:37.852868 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:37.852873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:37.852954 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:37.853025 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:37.854280 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.855342 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.857859 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.858362 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.858433 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.861973 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:37.861980 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:37.861985 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:37.862036 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:37.865584 (+ 3548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.866464 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.866470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.866473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.866626 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.867077 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.867082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.867086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.867461 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:37.867606 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.868120 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.868390 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607}
I20260807 19:42:37.869974 6847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.850896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:41979 (local address 127.30.208.1:48449)
0807 19:42:37.852502 (+ 1606us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:37.852523 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:37.852717 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:37.853276 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:37.853280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:37.853807 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:37.854103 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.854114 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.855514 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.855518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.857683 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.857697 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.858655 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.858659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.858796 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.859270 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:37.859286 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:37.861754 (+ 2468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:37.865750 (+ 3996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.865755 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.865759 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.866284 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.866771 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.866776 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.866780 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.866944 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.867692 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:37.867696 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:37.867846 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 19:42:37.869229 (+ 1383us) client_negotiation.cc:240] Negotiation successful
0807 19:42:37.869509 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1481,"mutex_wait_us":755,"thread_start_us":1312,"threads_started":1}
I20260807 19:42:37.871131 6836 heartbeater.cc:347] Connected to a master server at 127.30.208.60:41979
I20260807 19:42:37.871276 6836 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:37.871708 6836 heartbeater.cc:511] Master 127.30.208.60:41979 requested a full tablet report, sending...
I20260807 19:42:37.872200 6843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.822165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:40519 (local address 127.30.208.1:52243)
0807 19:42:37.850585 (+ 28420us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:37.850603 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:37.850719 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:37.851931 (+ 1212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:37.851936 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:37.852875 (+ 939us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:37.853156 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.853168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.854673 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.854678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.857535 (+ 2857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:37.857544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.861388 (+ 3844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.861395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:37.861867 (+ 472us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.862398 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:37.862419 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:37.865612 (+ 3193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:37.869482 (+ 3870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.869486 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.869489 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.869995 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.870458 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:37.870462 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:37.870465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:37.870609 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:37.871312 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:37.871317 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:37.871479 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 19:42:37.871589 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 19:42:37.871835 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28243,"mutex_wait_us":379,"thread_start_us":27640,"threads_started":1}
I20260807 19:42:37.872334 6840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.821549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:52243 (local address 127.30.208.61:40519)
0807 19:42:37.836990 (+ 15441us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:37.836998 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:37.851184 (+ 14186us) server_negotiation.cc:407] Connection header received
0807 19:42:37.851271 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:37.851276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:37.851368 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:37.851478 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:37.853375 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.854484 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.857665 (+ 3181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.858315 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.858409 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.865934 (+ 7525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:37.865941 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:37.865947 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:37.865990 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:37.869331 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.870137 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.870142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.870145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.870316 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.870734 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.870738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.870741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.871095 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:37.871222 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.871731 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.871964 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15259,"thread_start_us":15458,"threads_started":1}
I20260807 19:42:37.872804 6846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:37.824731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49660 (local address 127.30.208.2:38995)
0807 19:42:37.849756 (+ 25025us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:37.849767 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:37.849803 (+ 36us) server_negotiation.cc:407] Connection header received
0807 19:42:37.850084 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:37.850093 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:37.850376 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:37.850531 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:37.852824 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.854373 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.856146 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:37.857016 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:37.857129 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:37.864267 (+ 7138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:37.864284 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:37.864300 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:37.864343 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:37.868084 (+ 3741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.868991 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.869000 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.869003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.869170 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:37.869659 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:37.869664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:37.869667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:37.870069 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:37.870380 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:37.872171 (+ 1791us) server_negotiation.cc:299] Negotiation successful
0807 19:42:37.872429 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24796,"thread_start_us":25080,"threads_started":1}
I20260807 19:42:37.873224 6838 heartbeater.cc:347] Connected to a master server at 127.30.208.61:40519
I20260807 19:42:37.873356 6838 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:37.873730 6838 heartbeater.cc:511] Master 127.30.208.61:40519 requested a full tablet report, sending...
I20260807 19:42:37.873937 6636 ts_manager.cc:194] Registered new tserver with Master: 3a6ac8ef1cb744cbbf4d3916bf4d44ee (127.30.208.1:35269)
I20260807 19:42:37.874608 6126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:37.875182 6432 ts_manager.cc:194] Registered new tserver with Master: 3a6ac8ef1cb744cbbf4d3916bf4d44ee (127.30.208.1:35269)
I20260807 19:42:37.959401 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 5991
I20260807 19:42:37.976686 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:38995
--local_ip_for_outbound_sockets=127.30.208.2
--tserver_master_addrs=127.30.208.62:33613
--webserver_port=45631
--webserver_interface=127.30.208.2
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.2.keytab
--principal=oryx/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.208.62:33613,127.30.208.61:40519
--tserver_master_addrs=127.30.208.62:33613,127.30.208.61:40519,127.30.208.60:41979 with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:42:38.130776 6390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: NON_VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: true } } }
I20260807 19:42:38.131145 6390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:38.132695 6853 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d: attempting to promote NON_VOTER 61adb8b777d7419196442d1345e0dd55 to VOTER
I20260807 19:42:38.133498 6853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } }
I20260807 19:42:38.135782 6647 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 1 LEARNER]: Refusing update from remote peer 4b72ec6924c24c76b5bee353ab72410d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 19:42:38.135928 6444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 FOLLOWER]: Refusing update from remote peer 4b72ec6924c24c76b5bee353ab72410d: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 19:42:38.136358 6852 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [LEADER]: Connected to new peer: Peer: permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 19:42:38.136773 6853 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 19:42:38.139446 6852 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 61adb8b777d7419196442d1345e0dd55 (127.30.208.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } } }
I20260807 19:42:38.140172 6444 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 61adb8b777d7419196442d1345e0dd55 (127.30.208.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } } }
I20260807 19:42:38.144203 6852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } } }
I20260807 19:42:38.144574 6852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:38.144975 6852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } } }
I20260807 19:42:38.145265 6852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:38.149652 6857 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } } }
I20260807 19:42:38.150045 6857 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:38.150388 6856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } } }
I20260807 19:42:38.150666 6856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:38.151032 6647 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 61adb8b777d7419196442d1345e0dd55 (127.30.208.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } } }
I20260807 19:42:38.159740 6858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b72ec6924c24c76b5bee353ab72410d" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } } }
I20260807 19:42:38.160002 6858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 19:42:38.320551 6851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:38.320829 6851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:38.320950 6851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:38.328817 6851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:38.328926 6851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:38.329077 6851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:42:38.338742 6851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.2.keytab
--principal=oryx/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:38995
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=45631
--tserver_master_addrs=127.30.208.62:33613,127.30.208.61:40519,127.30.208.60:41979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:38.340021 6851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:38.341323 6851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:38.355006 6864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:38.355131 6865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:38.355541 6867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:38.355789 6851 server_base.cc:1034] running on GCE node
I20260807 19:42:38.356424 6851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:38.357540 6851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:38.358734 6851 hybrid_clock.cc:648] HybridClock initialized: now 1786131758358684 us; error 46 us; skew 500 ppm
I20260807 19:42:38.359031 6851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:38 dist-test-slave-vq2q krb5kdc[5729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131758, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:38.362035 6851 init.cc:377] Logged in from keytab as oryx/127.30.208.2@KRBTEST.COM (short username oryx)
I20260807 19:42:38.363363 6851 webserver.cc:460] Webserver started at http://127.30.208.2:45631/ using document root <none> and password file <none>
I20260807 19:42:38.364024 6851 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:38.364152 6851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:38.369560 6851 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:42:38.372234 6875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:38.373742 6851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:38.373896 6851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "ca911cece32a4d55ba20d1bb25ac8deb"
format_stamp: "Formatted at 2026-08-07 19:42:35 on dist-test-slave-vq2q"
server_key: "dff10d9a2f538e81c08ad07d04d3455c"
server_key_iv: "dda851c193cceedf31cb2024db33b748"
server_key_version: "encryptionkey@0"
I20260807 19:42:38.374858 6851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:38.391355 6851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:38.395543 6851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:38.395807 6851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:38.396907 6851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:38.398363 6851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:38.398452 6851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:38.398559 6851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:38.398627 6851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:38.426606 6851 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:38995
I20260807 19:42:38.426733 6996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:38995 every 8 connection(s)
I20260807 19:42:38.428155 6851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:38.430435 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 6851
I20260807 19:42:38.430719 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 6138
Aug 07 19:42:38 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.208.3@KRBTEST.COM
Aug 07 19:42:38 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131758, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
Aug 07 19:42:38 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131758, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.60@KRBTEST.COM
I20260807 19:42:38.481896 7003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.430867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:49527 (local address 127.30.208.60:41979)
0807 19:42:38.457302 (+ 26435us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.457308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.457329 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:42:38.457508 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.457513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.457597 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.457700 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:38.460270 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.461537 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.465282 (+ 3745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.465931 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.466023 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.474583 (+ 8560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:38.474590 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:38.474594 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:38.474634 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:38.478127 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:38.479086 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:38.479091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:38.479094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:38.479241 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:38.479793 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:38.479798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:38.479801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:38.480157 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:38.480294 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.481280 (+ 986us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.481505 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26273,"thread_start_us":26962,"threads_started":1}
I20260807 19:42:38.483094 7006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.431549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:41979 (local address 127.30.208.2:49527)
0807 19:42:38.452659 (+ 21110us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:38.452730 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:38.455496 (+ 2766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:38.458290 (+ 2794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:38.458304 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:38.459174 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:38.460037 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.460062 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.461777 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.461783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.465094 (+ 3311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.465106 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.468090 (+ 2984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.468096 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.468391 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.469693 (+ 1302us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:38.469724 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:38.474337 (+ 4613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:38.478290 (+ 3953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:38.478300 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:38.478328 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:38.478938 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:38.479390 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:38.479395 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:38.479398 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:38.479651 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:38.480400 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:38.480408 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:38.480858 (+ 450us) client_negotiation.cc:769] Sending connection context
0807 19:42:38.481114 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 19:42:38.481653 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20732,"thread_start_us":24774,"threads_started":1}
I20260807 19:42:38.483618 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.431618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50500 (local address 127.30.208.3:39325)
0807 19:42:38.456417 (+ 24799us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.456427 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.458096 (+ 1669us) server_negotiation.cc:407] Connection header received
0807 19:42:38.458184 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.458194 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.458457 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.458603 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:38.461070 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.462730 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.464988 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.465917 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.466042 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.474894 (+ 8852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:38.474917 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:38.474937 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:38.474992 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:38.479052 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:38.480032 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:38.480041 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:38.480044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:38.480201 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:38.480673 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:38.480678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:38.480681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:38.481121 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:38.481413 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.482593 (+ 1180us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.482860 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24629,"thread_start_us":24472,"threads_started":1}
I20260807 19:42:38.484467 7009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.437688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.2:47651)
0807 19:42:38.456716 (+ 19028us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:38.456735 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:38.456859 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:38.463901 (+ 7042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:38.463906 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:38.464570 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:38.464887 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.464901 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.466274 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.466279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.467820 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.467834 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.469092 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.469097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.469313 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.469821 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:38.469836 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:38.473540 (+ 3704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:38.479868 (+ 6328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:38.479872 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:38.479876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:38.480281 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:38.480834 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:38.480839 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:38.480842 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:38.481056 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:38.483630 (+ 2574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:38.483635 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:38.483787 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 19:42:38.483918 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 19:42:38.484152 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18903,"spinlock_wait_cycles":28928,"thread_start_us":18740,"threads_started":1}
I20260807 19:42:38.484624 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.437740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:47651 (local address 127.30.208.62:33613)
0807 19:42:38.451868 (+ 14128us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.451876 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.463573 (+ 11697us) server_negotiation.cc:407] Connection header received
0807 19:42:38.463654 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.463659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.463767 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.463885 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:38.465027 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.466161 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.468346 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.468930 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.469050 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.473822 (+ 4772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:38.473829 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:38.473834 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:38.473882 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:38.479744 (+ 5862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:38.480376 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:38.480382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:38.480385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:38.480702 (+ 317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:38.482582 (+ 1880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:38.482587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:38.482590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:38.483498 (+ 908us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:38.483642 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.484030 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.484261 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13969,"thread_start_us":14149,"threads_started":1}
I20260807 19:42:38.485277 6997 heartbeater.cc:347] Connected to a master server at 127.30.208.60:41979
I20260807 19:42:38.485574 6997 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:38.485846 6269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:38.486312 6999 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33613
I20260807 19:42:38.486464 6999 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:38.486619 6997 heartbeater.cc:511] Master 127.30.208.60:41979 requested a full tablet report, sending...
I20260807 19:42:38.486912 6999 heartbeater.cc:511] Master 127.30.208.62:33613 requested a full tablet report, sending...
Aug 07 19:42:38 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131758, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.61@KRBTEST.COM
I20260807 19:42:38.488451 5777 ts_manager.cc:194] Re-registered known tserver with Master: ca911cece32a4d55ba20d1bb25ac8deb (127.30.208.2:38995)
I20260807 19:42:38.489588 5777 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:47651
I20260807 19:42:38.490799 6636 ts_manager.cc:194] Registered new tserver with Master: ca911cece32a4d55ba20d1bb25ac8deb (127.30.208.2:38995)
I20260807 19:42:38.496170 7010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.457644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:58597 (local address 127.30.208.61:40519)
0807 19:42:38.473260 (+ 15616us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:38.473267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:38.475123 (+ 1856us) server_negotiation.cc:407] Connection header received
0807 19:42:38.475243 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:38.475249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:38.475332 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:38.475437 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:38.477310 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.478368 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.481218 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.481819 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.481951 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.490184 (+ 8233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:38.490191 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:38.490195 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:38.490277 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:38.493475 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:38.494243 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:38.494247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:38.494249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:38.494374 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:38.494853 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:38.494857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:38.494859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:38.495171 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:38.495295 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:38.495738 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 19:42:38.495925 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15462,"thread_start_us":16901,"threads_started":1}
I20260807 19:42:38.496308 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:38.464682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:40519 (local address 127.30.208.2:58597)
0807 19:42:38.474895 (+ 10213us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:38.474913 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:38.475024 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:38.475491 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:38.475495 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:38.476818 (+ 1323us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:38.477097 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.477110 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.478542 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.478547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.480652 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:38.480665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:38.482228 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:38.482244 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:38.482429 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:38.482973 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:38.482995 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:38.489926 (+ 6931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:38.493611 (+ 3685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:38.493617 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:38.493622 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:38.494123 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:38.494520 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:38.494525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:38.494529 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:38.494730 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:38.495270 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:38.495275 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:38.495450 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 19:42:38.495585 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 19:42:38.495932 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10083,"thread_start_us":8846,"threads_started":1}
I20260807 19:42:38.497241 7000 heartbeater.cc:347] Connected to a master server at 127.30.208.61:40519
I20260807 19:42:38.497371 7000 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:38.497789 7000 heartbeater.cc:511] Master 127.30.208.61:40519 requested a full tablet report, sending...
I20260807 19:42:38.498881 6432 ts_manager.cc:194] Registered new tserver with Master: ca911cece32a4d55ba20d1bb25ac8deb (127.30.208.2:38995)
I20260807 19:42:38.571552 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 6138
I20260807 19:42:38.588798 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:39325
--local_ip_for_outbound_sockets=127.30.208.3
--tserver_master_addrs=127.30.208.62:33613
--webserver_port=38615
--webserver_interface=127.30.208.3
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.3.keytab
--principal=oryx/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.208.62:33613,127.30.208.61:40519
--tserver_master_addrs=127.30.208.62:33613,127.30.208.61:40519,127.30.208.60:41979 with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:42:38.871747 6837 heartbeater.cc:503] Master 127.30.208.62:33613 was elected leader, sending a full tablet report...
W20260807 19:42:38.932827 7014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:38.933139 7014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:38.933306 7014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:38.940733 7014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:38.940876 7014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:38.941037 7014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:42:38.951102 7014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.3.keytab
--principal=oryx/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:39325
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=38615
--tserver_master_addrs=127.30.208.62:33613,127.30.208.61:40519,127.30.208.60:41979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:38.952661 7014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:38.954082 7014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:38.967137 7019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:38.967167 7022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:38.967095 7020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:38.967931 7014 server_base.cc:1034] running on GCE node
I20260807 19:42:38.968536 7014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:38.969507 7014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:38.970726 7014 hybrid_clock.cc:648] HybridClock initialized: now 1786131758970666 us; error 45 us; skew 500 ppm
I20260807 19:42:38.971062 7014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:38 dist-test-slave-vq2q krb5kdc[5729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131758, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:38.974233 7014 init.cc:377] Logged in from keytab as oryx/127.30.208.3@KRBTEST.COM (short username oryx)
I20260807 19:42:38.975740 7014 webserver.cc:460] Webserver started at http://127.30.208.3:38615/ using document root <none> and password file <none>
I20260807 19:42:38.976464 7014 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:38.976588 7014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:38.982350 7014 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:42:38.985338 7029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:38.986850 7014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:42:38.987013 7014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "132385f9496d4ecbb98ec6d6419b46dd"
format_stamp: "Formatted at 2026-08-07 19:42:35 on dist-test-slave-vq2q"
server_key: "e6882ec753f670d51ed3ab86dda9bbec"
server_key_iv: "340a3c8f0339e26be4eaf8ede488eeef"
server_key_version: "encryptionkey@0"
I20260807 19:42:38.987991 7014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:39.013465 7014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:39.018035 7014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:39.018301 7014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:39.019281 7014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:39.020699 7014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:39.020795 7014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:39.020877 7014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:39.020931 7014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:39.051848 7014 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:39325
I20260807 19:42:39.051932 7153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:39325 every 8 connection(s)
I20260807 19:42:39.053382 7014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:39.062814 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 7014
Aug 07 19:42:39 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131758, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
Aug 07 19:42:39 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131758, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.61@KRBTEST.COM
Aug 07 19:42:39 dist-test-slave-vq2q krb5kdc[5729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131758, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.60@KRBTEST.COM
I20260807 19:42:39.082300 7163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.056844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.3:59841)
0807 19:42:39.058189 (+ 1345us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:39.058235 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:39.060409 (+ 2174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:39.061128 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:39.061135 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:39.061996 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:39.062768 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.062790 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.063803 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.063808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.066544 (+ 2736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.066555 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.067763 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.067770 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.068376 (+ 606us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.069332 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:39.069353 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:39.072606 (+ 3253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:39.077026 (+ 4420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:39.077037 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:39.077058 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:39.077707 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:39.078206 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:39.078211 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:39.078215 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:39.078480 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:39.079306 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:39.079321 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:39.079852 (+ 531us) client_negotiation.cc:769] Sending connection context
0807 19:42:39.080189 (+ 337us) client_negotiation.cc:240] Negotiation successful
0807 19:42:39.080791 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1198,"mutex_wait_us":731,"thread_start_us":731,"threads_started":1}
I20260807 19:42:39.083523 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.056262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:59841 (local address 127.30.208.62:33613)
0807 19:42:39.057254 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.057260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.058365 (+ 1105us) server_negotiation.cc:407] Connection header received
0807 19:42:39.060696 (+ 2331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.060702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.060788 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.060871 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:39.062946 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.063656 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.066695 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.067366 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.067465 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.072955 (+ 5490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:39.072962 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:39.072967 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:39.073013 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:39.076844 (+ 3831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.077886 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.077892 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.077896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.078071 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.078611 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.078617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.078620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.079077 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:39.079226 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.082883 (+ 3657us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.083144 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842,"thread_start_us":745,"threads_started":1}
I20260807 19:42:39.084985 7155 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33613
I20260807 19:42:39.085395 7155 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:39.086397 7155 heartbeater.cc:511] Master 127.30.208.62:33613 requested a full tablet report, sending...
I20260807 19:42:39.088375 5777 ts_manager.cc:194] Re-registered known tserver with Master: 132385f9496d4ecbb98ec6d6419b46dd (127.30.208.3:39325)
I20260807 19:42:39.089533 5777 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:59841
I20260807 19:42:39.094667 7161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.056290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:56423 (local address 127.30.208.60:41979)
0807 19:42:39.057668 (+ 1378us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.057675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.058355 (+ 680us) server_negotiation.cc:407] Connection header received
0807 19:42:39.060680 (+ 2325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.060685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.060791 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.060886 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:39.064709 (+ 3823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.065895 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.067765 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.068946 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.069030 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.086181 (+ 17151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:39.086189 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:39.086193 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:39.086239 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:39.089923 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.091472 (+ 1549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.091478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.091481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.091655 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.092550 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.092556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.092559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.092958 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:39.093092 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.094073 (+ 981us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.094310 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1247,"thread_start_us":1116,"threads_started":1}
I20260807 19:42:39.096767 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.056844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:41979 (local address 127.30.208.3:56423)
0807 19:42:39.058184 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:39.058237 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:39.060411 (+ 2174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:39.061126 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:39.061134 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:39.062179 (+ 1045us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:39.064509 (+ 2330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.064526 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.066090 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.066095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.067608 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.067621 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.069313 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.069319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.069511 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.078934 (+ 9423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:39.078959 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:39.085147 (+ 6188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:39.090083 (+ 4936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:39.090088 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:39.090092 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:39.090642 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:39.091851 (+ 1209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:39.091856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:39.091859 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:39.092042 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:39.093203 (+ 1161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:39.093208 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:39.093355 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 19:42:39.095372 (+ 2017us) client_negotiation.cc:240] Negotiation successful
0807 19:42:39.095693 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1031,"mutex_wait_us":27,"spinlock_wait_cycles":47872,"thread_start_us":751,"threads_started":1}
I20260807 19:42:39.097747 7154 heartbeater.cc:347] Connected to a master server at 127.30.208.60:41979
I20260807 19:42:39.097908 7154 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:39.098311 7154 heartbeater.cc:511] Master 127.30.208.60:41979 requested a full tablet report, sending...
I20260807 19:42:39.099598 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.058893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:54889 (local address 127.30.208.61:40519)
0807 19:42:39.059739 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.059745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.059764 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:42:39.060757 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.060763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.060862 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.061298 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:39.062988 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.064091 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.066386 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.067095 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.069284 (+ 2189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.077407 (+ 8123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:39.077414 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:39.077420 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:39.077477 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:39.083148 (+ 5671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.093897 (+ 10749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.093903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.093907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.094094 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.095189 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.095194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.095197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.096108 (+ 911us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:39.098011 (+ 1903us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.098162 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.099224 (+ 1062us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":632,"threads_started":1}
I20260807 19:42:39.099794 6636 ts_manager.cc:194] Registered new tserver with Master: 132385f9496d4ecbb98ec6d6419b46dd (127.30.208.3:39325)
I20260807 19:42:39.100386 7164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.058769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:40519 (local address 127.30.208.3:54889)
0807 19:42:39.059316 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:39.059333 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:39.060420 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:39.061128 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:39.061135 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:39.062172 (+ 1037us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:39.062802 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.062817 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.064286 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.064292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.066221 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.066238 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.067402 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.067410 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.067797 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.068800 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:39.068833 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:39.074842 (+ 6009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:39.092958 (+ 18116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:39.092964 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:39.092967 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:39.093550 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:39.094524 (+ 974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:39.094529 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:39.094533 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:39.095005 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:39.096413 (+ 1408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:39.096418 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:39.096575 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 19:42:39.096712 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 19:42:39.097360 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":130,"thread_start_us":277,"threads_started":1}
I20260807 19:42:39.101336 7157 heartbeater.cc:347] Connected to a master server at 127.30.208.61:40519
I20260807 19:42:39.101476 7157 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:39.101843 7157 heartbeater.cc:511] Master 127.30.208.61:40519 requested a full tablet report, sending...
I20260807 19:42:39.102975 6432 ts_manager.cc:194] Registered new tserver with Master: 132385f9496d4ecbb98ec6d6419b46dd (127.30.208.3:39325)
I20260807 19:42:39.104629 7159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.070476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55584 (local address 127.30.208.62:33613)
0807 19:42:39.083779 (+ 13303us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.083785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.086617 (+ 2832us) server_negotiation.cc:407] Connection header received
0807 19:42:39.086904 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.086909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.086993 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.087065 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:39.089618 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.090822 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.093428 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.094067 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.094163 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.097853 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:39.097860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:39.097866 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:39.097908 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:39.101305 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.102242 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.102247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.102250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.102401 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.102873 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.102877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.102880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.103265 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:39.103492 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.104047 (+ 555us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.104280 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13183,"thread_start_us":30448,"threads_started":1}
I20260807 19:42:39.109473 5777 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:39.118029 7173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.071115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59168 (local address 127.30.208.61:40519)
0807 19:42:39.086002 (+ 14887us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.086009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.101745 (+ 15736us) server_negotiation.cc:407] Connection header received
0807 19:42:39.101928 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.101933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.102016 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.102113 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:39.104651 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.105899 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.107863 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.108360 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.108431 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.111446 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:39.111453 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:39.111458 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:39.111509 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:39.114959 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.115782 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.115788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.115792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.115963 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.116412 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.116417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.116420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.116778 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:39.117052 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.117409 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.117637 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14740,"thread_start_us":14594,"threads_started":1}
W20260807 19:42:39.119480 5777 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:39.120108 7161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.086785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35412 (local address 127.30.208.60:41979)
0807 19:42:39.095304 (+ 8519us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.095310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.104516 (+ 9206us) server_negotiation.cc:407] Connection header received
0807 19:42:39.105284 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.105291 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.105389 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.105486 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:39.108068 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.108910 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.111215 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.111722 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.111834 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.114344 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:39.114349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:39.114352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:39.114384 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:39.116990 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.117830 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.117836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.117840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.117995 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.118449 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.118454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.118457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.118809 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:39.119027 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.119576 (+ 549us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.119798 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8399,"thread_start_us":19664,"threads_started":1}
I20260807 19:42:39.153453 7188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.139370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:39325 (local address 127.0.0.1:50510)
0807 19:42:39.145281 (+ 5911us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:39.145297 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:39.145439 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:39.146136 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:39.146141 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:39.146167 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:39.146499 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.146510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.148739 (+ 2229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.148745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.150808 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.150824 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.152656 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.152662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.152827 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.152864 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:42:39.152945 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 19:42:39.153046 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5767,"mutex_wait_us":1,"thread_start_us":5722,"threads_started":1}
I20260807 19:42:39.153513 7189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.139599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50510 (local address 127.30.208.3:39325)
0807 19:42:39.140385 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.140393 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.145417 (+ 5024us) server_negotiation.cc:407] Connection header received
0807 19:42:39.145609 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.145618 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.145949 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.146159 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:39.146662 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.148614 (+ 1952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.150975 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.152479 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.152605 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.152828 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.152998 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.153168 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":466,"threads_started":1}
I20260807 19:42:39.154470 7186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.138880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:35269 (local address 127.0.0.1:44494)
0807 19:42:39.139646 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:39.139678 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:39.139828 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:39.147100 (+ 7272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:39.147106 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:39.147138 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:39.147715 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.147736 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.149258 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.149264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.151400 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.151413 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.153891 (+ 2478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.153898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.154060 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.154084 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:42:39.154136 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 19:42:39.154229 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":282,"threads_started":1}
I20260807 19:42:39.155484 7187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.139151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44494 (local address 127.30.208.1:35269)
0807 19:42:39.145943 (+ 6792us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.145952 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.145978 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:42:39.146627 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.146644 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.146878 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.147082 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:39.147956 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.149104 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.151546 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.153648 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.154706 (+ 1058us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.154931 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.155026 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.155197 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6626,"thread_start_us":6498,"threads_started":1}
I20260807 19:42:39.157724 7082 tablet_service.cc:1467] Processing CreateTablet for tablet a24d5101a8f3451f952b5787c97aa64c (DEFAULT_TABLE table=test-table [id=5b0c61812bdb4680b40e9c1537821032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:39.159164 7082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a24d5101a8f3451f952b5787c97aa64c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:39.160813 6761 tablet_service.cc:1467] Processing CreateTablet for tablet a24d5101a8f3451f952b5787c97aa64c (DEFAULT_TABLE table=test-table [id=5b0c61812bdb4680b40e9c1537821032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:39.162190 6761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a24d5101a8f3451f952b5787c97aa64c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:39.171595 7191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.145512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:38995 (local address 127.0.0.1:49674)
0807 19:42:39.147834 (+ 2322us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:39.147861 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:39.147972 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:39.149737 (+ 1765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:39.149742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:39.149767 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:39.150195 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.150208 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.164586 (+ 14378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.164592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.166681 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.166701 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.170715 (+ 4014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.170723 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.170930 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.170966 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:42:39.171064 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 19:42:39.171222 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2194,"thread_start_us":1474,"threads_started":1}
I20260807 19:42:39.172755 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.145770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49674 (local address 127.30.208.2:38995)
0807 19:42:39.146283 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.146291 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.149282 (+ 2991us) server_negotiation.cc:407] Connection header received
0807 19:42:39.149379 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.149384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.149545 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.149724 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:39.162217 (+ 12493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.164364 (+ 2147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.166917 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.170178 (+ 3261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.170329 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.170544 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.172285 (+ 1741us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.172467 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":369,"threads_started":1}
I20260807 19:42:39.174870 7192 tablet_bootstrap.cc:492] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd: Bootstrap starting.
I20260807 19:42:39.176586 6928 tablet_service.cc:1467] Processing CreateTablet for tablet a24d5101a8f3451f952b5787c97aa64c (DEFAULT_TABLE table=test-table [id=5b0c61812bdb4680b40e9c1537821032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:39.178386 6928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a24d5101a8f3451f952b5787c97aa64c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:39.179131 7192 tablet_bootstrap.cc:654] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:39.180548 7192 log.cc:824] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:39.185539 7192 tablet_bootstrap.cc:492] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd: No bootstrap required, opened a new log
I20260807 19:42:39.185765 7193 tablet_bootstrap.cc:492] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Bootstrap starting.
I20260807 19:42:39.185889 7192 ts_tablet_manager.cc:1397] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260807 19:42:39.189610 7193 tablet_bootstrap.cc:654] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:39.190958 7193 log.cc:824] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:39.194327 7193 tablet_bootstrap.cc:492] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: No bootstrap required, opened a new log
I20260807 19:42:39.194669 7193 ts_tablet_manager.cc:1397] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260807 19:42:39.194604 7192 raft_consensus.cc:348] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:39.195031 7192 raft_consensus.cc:374] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:39.195211 7192 raft_consensus.cc:729] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 132385f9496d4ecbb98ec6d6419b46dd, State: Initialized, Role: FOLLOWER
I20260807 19:42:39.195830 7196 tablet_bootstrap.cc:492] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb: Bootstrap starting.
I20260807 19:42:39.195878 7192 consensus_queue.cc:260] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:39.197994 7192 ts_tablet_manager.cc:1428] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260807 19:42:39.198277 7155 heartbeater.cc:503] Master 127.30.208.62:33613 was elected leader, sending a full tablet report...
I20260807 19:42:39.200232 7196 tablet_bootstrap.cc:654] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:39.201690 7196 log.cc:824] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:39.205842 7193 raft_consensus.cc:348] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:39.206316 7193 raft_consensus.cc:374] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:39.206436 7193 raft_consensus.cc:729] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a6ac8ef1cb744cbbf4d3916bf4d44ee, State: Initialized, Role: FOLLOWER
I20260807 19:42:39.207080 7193 consensus_queue.cc:260] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:39.207547 7196 tablet_bootstrap.cc:492] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb: No bootstrap required, opened a new log
I20260807 19:42:39.207861 7196 ts_tablet_manager.cc:1397] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260807 19:42:39.208904 7193 ts_tablet_manager.cc:1428] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Time spent starting tablet: real 0.014s user 0.010s sys 0.002s
I20260807 19:42:39.216660 7196 raft_consensus.cc:348] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:39.217029 7196 raft_consensus.cc:374] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:39.217131 7196 raft_consensus.cc:729] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca911cece32a4d55ba20d1bb25ac8deb, State: Initialized, Role: FOLLOWER
I20260807 19:42:39.217743 7196 consensus_queue.cc:260] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:39.218681 6999 heartbeater.cc:503] Master 127.30.208.62:33613 was elected leader, sending a full tablet report...
I20260807 19:42:39.219208 7196 ts_tablet_manager.cc:1428] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb: Time spent starting tablet: real 0.011s user 0.004s sys 0.008s
W20260807 19:42:39.226948 7001 tablet.cc:2369] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:39.231542 7200 raft_consensus.cc:482] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:39.231884 7200 raft_consensus.cc:504] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:39.233577 7200 leader_election.cc:288] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 132385f9496d4ecbb98ec6d6419b46dd (127.30.208.3:39325), ca911cece32a4d55ba20d1bb25ac8deb (127.30.208.2:38995)
I20260807 19:42:39.240192 7202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.233889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:39325 (local address 127.30.208.1:55901)
0807 19:42:39.234567 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:39.234584 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:39.234726 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:39.235036 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:39.235040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:39.235078 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:39.235443 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.235454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.236458 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.236462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.238007 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.238022 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.239594 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.239599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.239713 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.239737 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:42:39.239810 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 19:42:39.239910 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":2,"thread_start_us":362,"threads_started":1}
I20260807 19:42:39.240226 7189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.233902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:55901 (local address 127.30.208.3:39325)
0807 19:42:39.234128 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.234135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.234674 (+ 539us) server_negotiation.cc:407] Connection header received
0807 19:42:39.234859 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.234863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.234930 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.235007 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:39.235583 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.236324 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.238203 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.239354 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.239519 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.239594 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.239849 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.239987 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:42:39.241297 7190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.234783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:42355 (local address 127.30.208.2:38995)
0807 19:42:39.234996 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.235002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.235596 (+ 594us) server_negotiation.cc:407] Connection header received
0807 19:42:39.235750 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.235753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.235811 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.235885 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:39.236355 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.237024 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.239171 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.240302 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.240471 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.240527 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.240911 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.241037 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:42:39.241377 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.234658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:38995 (local address 127.30.208.1:42355)
0807 19:42:39.235492 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:39.235506 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:39.235627 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:39.235917 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:39.235921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:39.235945 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:39.236210 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.236218 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.237158 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.237161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.238984 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:39.239000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.240644 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.240649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:39.240759 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.240780 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 19:42:39.240856 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 19:42:39.240973 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":1,"thread_start_us":552,"threads_started":1}
I20260807 19:42:39.241164 7105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24d5101a8f3451f952b5787c97aa64c" candidate_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "132385f9496d4ecbb98ec6d6419b46dd" is_pre_election: true
I20260807 19:42:39.241734 7105 raft_consensus.cc:2454] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a6ac8ef1cb744cbbf4d3916bf4d44ee in term 0.
I20260807 19:42:39.242341 6710 leader_election.cc:302] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 132385f9496d4ecbb98ec6d6419b46dd, 3a6ac8ef1cb744cbbf4d3916bf4d44ee; no voters:
I20260807 19:42:39.242784 7200 raft_consensus.cc:2781] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:39.242949 7200 raft_consensus.cc:482] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:39.242842 6949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24d5101a8f3451f952b5787c97aa64c" candidate_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" is_pre_election: true
I20260807 19:42:39.243043 7200 raft_consensus.cc:3037] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:39.243310 6949 raft_consensus.cc:2454] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a6ac8ef1cb744cbbf4d3916bf4d44ee in term 0.
I20260807 19:42:39.245355 7200 raft_consensus.cc:504] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:39.245903 7200 leader_election.cc:288] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [CANDIDATE]: Term 1 election: Requested vote from peers 132385f9496d4ecbb98ec6d6419b46dd (127.30.208.3:39325), ca911cece32a4d55ba20d1bb25ac8deb (127.30.208.2:38995)
I20260807 19:42:39.246384 6949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24d5101a8f3451f952b5787c97aa64c" candidate_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca911cece32a4d55ba20d1bb25ac8deb"
I20260807 19:42:39.246364 7105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a24d5101a8f3451f952b5787c97aa64c" candidate_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "132385f9496d4ecbb98ec6d6419b46dd"
I20260807 19:42:39.246592 6949 raft_consensus.cc:3037] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:39.246599 7105 raft_consensus.cc:3037] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:39.249785 7105 raft_consensus.cc:2454] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a6ac8ef1cb744cbbf4d3916bf4d44ee in term 1.
I20260807 19:42:39.249785 6949 raft_consensus.cc:2454] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a6ac8ef1cb744cbbf4d3916bf4d44ee in term 1.
I20260807 19:42:39.250152 6710 leader_election.cc:302] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 132385f9496d4ecbb98ec6d6419b46dd, 3a6ac8ef1cb744cbbf4d3916bf4d44ee; no voters:
I20260807 19:42:39.250447 7200 raft_consensus.cc:2781] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:39.250874 7200 raft_consensus.cc:686] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 1 LEADER]: Becoming Leader. State: Replica: 3a6ac8ef1cb744cbbf4d3916bf4d44ee, State: Running, Role: LEADER
I20260807 19:42:39.251395 7200 consensus_queue.cc:237] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:39.256472 5776 catalog_manager.cc:5495] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a6ac8ef1cb744cbbf4d3916bf4d44ee (127.30.208.1). New cstate: current_term: 1 leader_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } health_report { overall_health: UNKNOWN } } }
I20260807 19:42:39.286780 7187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.281617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44502 (local address 127.30.208.1:35269)
0807 19:42:39.281833 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.281838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.281854 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:42:39.281976 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.281981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.282065 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.282201 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:39.282557 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.283308 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.284651 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.285070 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.285199 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.285682 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:39.286192 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:39.286326 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.286432 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.286515 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:42:39.295135 6949 raft_consensus.cc:1261] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb [term 1 FOLLOWER]: Refusing update from remote peer 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:39.295135 7105 raft_consensus.cc:1261] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd [term 1 FOLLOWER]: Refusing update from remote peer 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:39.296193 7200 consensus_queue.cc:1035] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:39.296587 7204 consensus_queue.cc:1035] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:42:39.305719 7158 tablet.cc:2369] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:39.308537 7207 mvcc.cc:205] Tried to move back new op lower bound from 7315995686062682112 to 7315995685897625600. Current Snapshot: MvccSnapshot[applied={T|T < 7315995686062682112}]
I20260807 19:42:39.310674 7209 mvcc.cc:205] Tried to move back new op lower bound from 7315995686062682112 to 7315995685897625600. Current Snapshot: MvccSnapshot[applied={T|T < 7315995686062682112}]
I20260807 19:42:39.326267 7210 mvcc.cc:205] Tried to move back new op lower bound from 7315995686062682112 to 7315995685897625600. Current Snapshot: MvccSnapshot[applied={T|T < 7315995686062682112 or (T in {7315995686062682112})}]
I20260807 19:42:39.329229 5777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:39.329581 5777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:39.337267 5777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b72ec6924c24c76b5bee353ab72410d: Sending DeleteTablet for 3 replicas of tablet a24d5101a8f3451f952b5787c97aa64c
I20260807 19:42:39.338500 7082 tablet_service.cc:1514] Processing DeleteTablet for tablet a24d5101a8f3451f952b5787c97aa64c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:39 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:50510
I20260807 19:42:39.338792 6761 tablet_service.cc:1514] Processing DeleteTablet for tablet a24d5101a8f3451f952b5787c97aa64c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:39 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:44494
I20260807 19:42:39.338789 6926 tablet_service.cc:1514] Processing DeleteTablet for tablet a24d5101a8f3451f952b5787c97aa64c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:39 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:49674
I20260807 19:42:39.339363 6928 tablet_service.cc:1514] Processing DeleteTablet for tablet a24d5101a8f3451f952b5787c97aa64c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:39 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:49674
I20260807 19:42:39.339416 7217 tablet_replica.cc:330] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd: stopping tablet replica
I20260807 19:42:39.339895 7217 raft_consensus.cc:2229] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:39.340452 7217 raft_consensus.cc:2258] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:39.349511 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 5743
I20260807 19:42:39.350672 5809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:39.353255 7217 ts_tablet_manager.cc:1905] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:39.354281 7219 tablet_replica.cc:330] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb: stopping tablet replica
I20260807 19:42:39.354760 7219 raft_consensus.cc:2229] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:39.355281 7219 raft_consensus.cc:2258] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:39.356077 7218 tablet_replica.cc:330] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: stopping tablet replica
I20260807 19:42:39.356499 7218 raft_consensus.cc:2229] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:39.357115 7218 raft_consensus.cc:2258] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:39.361991 7217 ts_tablet_manager.cc:1918] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:39.362195 7217 log.cc:1196] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/a24d5101a8f3451f952b5787c97aa64c
I20260807 19:42:39.362958 7217 ts_tablet_manager.cc:1939] T a24d5101a8f3451f952b5787c97aa64c P 132385f9496d4ecbb98ec6d6419b46dd: Deleting consensus metadata
I20260807 19:42:39.364746 7218 ts_tablet_manager.cc:1905] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:39.366874 7219 ts_tablet_manager.cc:1905] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:39.369755 7173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.342224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59178 (local address 127.30.208.61:40519)
0807 19:42:39.342416 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.342422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.348190 (+ 5768us) server_negotiation.cc:407] Connection header received
0807 19:42:39.349296 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.349304 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.349418 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.349494 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:39.354356 (+ 4862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.355416 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.357460 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.358161 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.358268 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.362387 (+ 4119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:39.362395 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:39.362399 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:39.362444 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:39.366292 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.367282 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.367287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.367290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.367455 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.367934 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.367939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.367943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.368286 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:39.368528 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.369130 (+ 602us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.369420 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:42:39.381538 7218 ts_tablet_manager.cc:1918] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:39.381919 7218 log.cc:1196] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/a24d5101a8f3451f952b5787c97aa64c
I20260807 19:42:39.383158 7219 ts_tablet_manager.cc:1918] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:39.383381 7219 log.cc:1196] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/a24d5101a8f3451f952b5787c97aa64c
I20260807 19:42:39.384176 7219 ts_tablet_manager.cc:1939] T a24d5101a8f3451f952b5787c97aa64c P ca911cece32a4d55ba20d1bb25ac8deb: Deleting consensus metadata
I20260807 19:42:39.387238 7218 ts_tablet_manager.cc:1939] T a24d5101a8f3451f952b5787c97aa64c P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Deleting consensus metadata
I20260807 19:42:39.393478 7161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.348417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35422 (local address 127.30.208.60:41979)
0807 19:42:39.348586 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.348592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.372073 (+ 23481us) server_negotiation.cc:407] Connection header received
0807 19:42:39.372240 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.372245 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.372357 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.372449 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:39.374217 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.375487 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.377946 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.378867 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.379012 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.383794 (+ 4782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:39.383803 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:39.383808 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:39.383860 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:39.388773 (+ 4913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.389996 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.390003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.390007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.390252 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.390970 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.390975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.390979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.391522 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:39.391841 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.392575 (+ 734us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.392816 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:42:39.510470 5762 catalog_manager.cc:4841] TS 3a6ac8ef1cb744cbbf4d3916bf4d44ee (127.30.208.1:35269): tablet a24d5101a8f3451f952b5787c97aa64c (table test-table [id=5b0c61812bdb4680b40e9c1537821032]) successfully deleted
I20260807 19:42:39.510618 5761 catalog_manager.cc:4841] TS ca911cece32a4d55ba20d1bb25ac8deb (127.30.208.2:38995): tablet a24d5101a8f3451f952b5787c97aa64c (table test-table [id=5b0c61812bdb4680b40e9c1537821032]) successfully deleted
W20260807 19:42:39.511153 5761 catalog_manager.cc:4808] TS ca911cece32a4d55ba20d1bb25ac8deb (127.30.208.2:38995): delete failed for tablet a24d5101a8f3451f952b5787c97aa64c because the tablet was not found. No further retry: Not found: Tablet not found: a24d5101a8f3451f952b5787c97aa64c
I20260807 19:42:39.511436 5761 catalog_manager.cc:4841] TS 132385f9496d4ecbb98ec6d6419b46dd (127.30.208.3:39325): tablet a24d5101a8f3451f952b5787c97aa64c (table test-table [id=5b0c61812bdb4680b40e9c1537821032]) successfully deleted
I20260807 19:42:39.511925 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 5743
I20260807 19:42:39.555687 7173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.538415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59184 (local address 127.30.208.61:40519)
0807 19:42:39.538629 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.538636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.539631 (+ 995us) server_negotiation.cc:407] Connection header received
0807 19:42:39.539917 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.539923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.540012 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.540081 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:39.542751 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.543916 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.545633 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.546112 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.546236 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.549602 (+ 3366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:39.549608 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:39.549613 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:39.549650 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:39.552773 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.553510 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.553516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.553520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.553677 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.554055 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.554060 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.554062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.554460 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:39.554712 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.555110 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.555342 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:42:39.555935 7161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:39.539879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35428 (local address 127.30.208.60:41979)
0807 19:42:39.540084 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:39.540090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:39.540109 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:42:39.540175 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:39.540181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:39.540260 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:39.540366 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:39.542784 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.543565 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.545363 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:39.546045 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:39.546191 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:39.548916 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:39.548921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:39.548924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:39.548963 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:39.552342 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.553214 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.553220 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.553224 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.553379 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:39.554142 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:39.554148 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:39.554151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:39.554497 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:39.554711 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:39.555246 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 19:42:39.555627 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:42:40.320031 7229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.319146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.3:56135)
0807 19:42:40.319657 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.319801 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":242,"threads_started":1}
W20260807 19:42:40.320605 7155 heartbeater.cc:644] Failed to heartbeat to 127.30.208.62:33613 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
I20260807 19:42:40.322383 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.321151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.1:44651)
0807 19:42:40.322041 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.322185 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":778,"thread_start_us":637,"threads_started":1}
W20260807 19:42:40.322754 6837 heartbeater.cc:644] Failed to heartbeat to 127.30.208.62:33613 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
I20260807 19:42:40.341360 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.340540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.2:46013)
0807 19:42:40.341053 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.341163 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":314,"threads_started":1}
W20260807 19:42:40.341730 6999 heartbeater.cc:644] Failed to heartbeat to 127.30.208.62:33613 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
I20260807 19:42:40.837822 7232 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b72ec6924c24c76b5bee353ab72410d)
I20260807 19:42:40.838138 7232 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } }
I20260807 19:42:40.838253 7233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4b72ec6924c24c76b5bee353ab72410d)
I20260807 19:42:40.838519 7233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } }
I20260807 19:42:40.839591 7232 leader_election.cc:288] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b72ec6924c24c76b5bee353ab72410d (127.30.208.62:33613), 61adb8b777d7419196442d1345e0dd55 (127.30.208.60:41979)
I20260807 19:42:40.839761 7233 leader_election.cc:288] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4b72ec6924c24c76b5bee353ab72410d (127.30.208.62:33613), f1aaac13b4e84290b4e10e03dbafc748 (127.30.208.61:40519)
I20260807 19:42:40.853117 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.839854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:54028)
0807 19:42:40.852530 (+ 12676us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.852803 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12509,"threads_started":1}
I20260807 19:42:40.855343 7234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.839793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:54026)
0807 19:42:40.854795 (+ 15002us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.855005 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14874,"thread_start_us":14695,"threads_started":1}
W20260807 19:42:40.855666 6416 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
W20260807 19:42:40.857664 6617 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
I20260807 19:42:40.862957 7234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.862544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:54048)
0807 19:42:40.862703 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.862796 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 19:42:40.863283 6416 leader_election.cc:334] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b72ec6924c24c76b5bee353ab72410d (127.30.208.62:33613): Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
I20260807 19:42:40.864388 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.840272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:40519 (local address 127.0.0.1:41322)
0807 19:42:40.853392 (+ 13120us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.853412 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:40.853602 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:40.855102 (+ 1500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:40.855110 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:40.855152 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:40.855785 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:40.855804 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.858337 (+ 2533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.858344 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:40.860797 (+ 2453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:40.860814 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.862811 (+ 1997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.862818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:40.863010 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:40.863683 (+ 673us) client_negotiation.cc:769] Sending connection context
0807 19:42:40.863788 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 19:42:40.863981 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13008,"mutex_wait_us":3,"thread_start_us":13437,"threads_started":1}
I20260807 19:42:40.865151 7237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.841005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41322 (local address 127.30.208.61:40519)
0807 19:42:40.850538 (+ 9533us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:40.850545 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:40.853524 (+ 2979us) server_negotiation.cc:407] Connection header received
0807 19:42:40.853749 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:40.853755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:40.853843 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:40.853956 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:40.856189 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.858085 (+ 1896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.861003 (+ 2918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.862536 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.863199 (+ 663us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:40.863320 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:40.864673 (+ 1353us) server_negotiation.cc:299] Negotiation successful
0807 19:42:40.864819 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":9393,"thread_start_us":9191,"threads_started":1}
I20260807 19:42:40.865753 6444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61adb8b777d7419196442d1345e0dd55" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f1aaac13b4e84290b4e10e03dbafc748" is_pre_election: true
I20260807 19:42:40.866137 6444 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61adb8b777d7419196442d1345e0dd55 in term 1.
I20260807 19:42:40.866496 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.865987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:54054)
0807 19:42:40.866196 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.866301 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260807 19:42:40.866588 7238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.841153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59798 (local address 127.30.208.60:41979)
0807 19:42:40.854308 (+ 13155us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:40.854314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:40.856239 (+ 1925us) server_negotiation.cc:407] Connection header received
0807 19:42:40.856545 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:40.856551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:40.856638 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:40.856707 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:40.858475 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.859785 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.863657 (+ 3872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.865159 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.865300 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:40.865414 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:40.866074 (+ 660us) server_negotiation.cc:299] Negotiation successful
0807 19:42:40.866213 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":13019,"thread_start_us":12881,"threads_started":1}
W20260807 19:42:40.866720 6617 leader_election.cc:334] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4b72ec6924c24c76b5bee353ab72410d (127.30.208.62:33613): Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
I20260807 19:42:40.867107 6619 leader_election.cc:302] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61adb8b777d7419196442d1345e0dd55, f1aaac13b4e84290b4e10e03dbafc748; no voters: 4b72ec6924c24c76b5bee353ab72410d
I20260807 19:42:40.867228 7239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.847368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:41979 (local address 127.0.0.1:59798)
0807 19:42:40.855091 (+ 7723us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.855109 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:40.856372 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:40.856974 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:40.856979 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:40.857008 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:40.857507 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:40.857520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.859976 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.859982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:40.863377 (+ 3395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:40.863395 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.865696 (+ 2301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.865702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:40.865883 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:40.865916 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:42:40.866743 (+ 827us) client_negotiation.cc:240] Negotiation successful
0807 19:42:40.866888 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7611,"mutex_wait_us":1293,"thread_start_us":8116,"threads_started":1}
I20260807 19:42:40.867568 7233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 19:42:40.867690 7233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4b72ec6924c24c76b5bee353ab72410d)
I20260807 19:42:40.867805 7233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:42:40.868067 6647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f1aaac13b4e84290b4e10e03dbafc748" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "61adb8b777d7419196442d1345e0dd55" is_pre_election: true
I20260807 19:42:40.870710 7233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } }
I20260807 19:42:40.871165 6647 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f1aaac13b4e84290b4e10e03dbafc748 in current term 2: Already voted for candidate 61adb8b777d7419196442d1345e0dd55 in this term.
I20260807 19:42:40.871531 7233 leader_election.cc:288] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [CANDIDATE]: Term 2 election: Requested vote from peers 4b72ec6924c24c76b5bee353ab72410d (127.30.208.62:33613), f1aaac13b4e84290b4e10e03dbafc748 (127.30.208.61:40519)
I20260807 19:42:40.872303 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.871753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:54064)
0807 19:42:40.872022 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.872120 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260807 19:42:40.872318 6444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "61adb8b777d7419196442d1345e0dd55" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f1aaac13b4e84290b4e10e03dbafc748"
I20260807 19:42:40.872610 6444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:42:40.872826 6418 leader_election.cc:302] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1aaac13b4e84290b4e10e03dbafc748; no voters: 4b72ec6924c24c76b5bee353ab72410d, 61adb8b777d7419196442d1345e0dd55
I20260807 19:42:40.873589 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.873083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:54066)
0807 19:42:40.873337 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.873434 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260807 19:42:40.873785 6617 leader_election.cc:334] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4b72ec6924c24c76b5bee353ab72410d (127.30.208.62:33613): Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
I20260807 19:42:40.875133 6444 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61adb8b777d7419196442d1345e0dd55 in term 2.
I20260807 19:42:40.875304 7232 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:42:40.875617 6619 leader_election.cc:302] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 61adb8b777d7419196442d1345e0dd55, f1aaac13b4e84290b4e10e03dbafc748; no voters: 4b72ec6924c24c76b5bee353ab72410d
I20260807 19:42:40.875972 7233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:42:40.876446 7233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [term 2 LEADER]: Becoming Leader. State: Replica: 61adb8b777d7419196442d1345e0dd55, State: Running, Role: LEADER
I20260807 19:42:40.876911 7233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } }
I20260807 19:42:40.879393 7243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61adb8b777d7419196442d1345e0dd55. Latest consensus state: current_term: 2 leader_uuid: "61adb8b777d7419196442d1345e0dd55" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } } }
I20260807 19:42:40.879799 7243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:40.880578 7245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:40.885823 7245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:40.887017 7245 catalog_manager.cc:1259] Loaded cluster ID: 48ade388e5c844bf927e36d1c2fe5952
I20260807 19:42:40.887115 7245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:40.888954 7245 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:40.890127 7245 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: Loaded TSK: 0
I20260807 19:42:40.891441 7245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:42:40.914261 6636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:40.921489 6636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:40.925653 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.925238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:54082)
0807 19:42:40.925403 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.925509 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 19:42:40.925819 6444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [term 2 FOLLOWER]: Refusing update from remote peer 61adb8b777d7419196442d1345e0dd55: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 19:42:40.926738 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.926279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:54098)
0807 19:42:40.926458 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.926562 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 19:42:40.926666 7243 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260807 19:42:40.926970 6617 consensus_peers.cc:487] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 -> Peer 4b72ec6924c24c76b5bee353ab72410d (127.30.208.62:33613): Couldn't send request to peer 4b72ec6924c24c76b5bee353ab72410d. Status: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 19:42:40.929448 7232 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61adb8b777d7419196442d1345e0dd55. Latest consensus state: current_term: 2 leader_uuid: "61adb8b777d7419196442d1345e0dd55" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } } }
I20260807 19:42:40.929718 7232 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:40.931253 7243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "61adb8b777d7419196442d1345e0dd55" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } } }
I20260807 19:42:40.931541 7243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:40.934996 7247 mvcc.cc:205] Tried to move back new op lower bound from 7315995692745388032 to 7315995692556509184. Current Snapshot: MvccSnapshot[applied={T|T < 7315995692745388032}]
I20260807 19:42:40.937752 7232 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "61adb8b777d7419196442d1345e0dd55" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "4b72ec6924c24c76b5bee353ab72410d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33613 } } peers { permanent_uuid: "f1aaac13b4e84290b4e10e03dbafc748" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 40519 } attrs { promote: false } } peers { permanent_uuid: "61adb8b777d7419196442d1345e0dd55" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 41979 } attrs { promote: false } } }
I20260807 19:42:40.938046 7232 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1aaac13b4e84290b4e10e03dbafc748 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:42:40.948858 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.942482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:35269 (local address 127.0.0.1:51644)
0807 19:42:40.942777 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.942797 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:40.942911 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:40.943327 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:40.943332 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:40.943358 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:40.943540 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:40.943549 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.945256 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.945261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:40.946918 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:40.946930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.948250 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.948256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:40.948433 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:40.948461 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 19:42:40.948526 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 19:42:40.948624 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260807 19:42:40.949712 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.942600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51644 (local address 127.30.208.1:35269)
0807 19:42:40.943048 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:40.943053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:40.943069 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:42:40.943127 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:40.943130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:40.943186 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:40.943281 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:40.944172 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.945090 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.947079 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.948078 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.949230 (+ 1152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:40.949314 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:40.949372 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 19:42:40.949476 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":238,"threads_started":1}
I20260807 19:42:40.950563 6761 tablet_service.cc:1467] Processing CreateTablet for tablet 86677a320afa41ba8e34e68fd005f48f (DEFAULT_TABLE table=test-table [id=ca277f2a28094a7eb25ebc7e3301e69a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:40.951102 6761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86677a320afa41ba8e34e68fd005f48f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:40.951601 7254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.943664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:38995 (local address 127.0.0.1:52952)
0807 19:42:40.944254 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.944270 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:40.944406 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:40.944728 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:40.944733 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:40.944755 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:40.945122 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:40.945133 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.946566 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.946572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:40.949541 (+ 2969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:40.949558 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.950890 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.950897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:40.951046 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:40.951070 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:42:40.951150 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 19:42:40.951281 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":504,"threads_started":1}
I20260807 19:42:40.952176 7255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.943838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52952 (local address 127.30.208.2:38995)
0807 19:42:40.944283 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:40.944288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:40.944331 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:42:40.944533 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:40.944538 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:40.944612 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:40.944713 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:40.945349 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.946422 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.949729 (+ 3307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.950675 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.950799 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:40.950895 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:40.951200 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 19:42:40.951345 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":216,"threads_started":1}
I20260807 19:42:40.952919 6926 tablet_service.cc:1467] Processing CreateTablet for tablet 86677a320afa41ba8e34e68fd005f48f (DEFAULT_TABLE table=test-table [id=ca277f2a28094a7eb25ebc7e3301e69a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:40.953361 6926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86677a320afa41ba8e34e68fd005f48f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:40.959036 7257 tablet_bootstrap.cc:492] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb: Bootstrap starting.
I20260807 19:42:40.960575 7257 tablet_bootstrap.cc:654] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:40.962543 7257 tablet_bootstrap.cc:492] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb: No bootstrap required, opened a new log
I20260807 19:42:40.962684 7257 ts_tablet_manager.cc:1397] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 19:42:40.963670 7256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.944055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49660 (local address 127.30.208.3:39325)
0807 19:42:40.956722 (+ 12667us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:40.956729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:40.956751 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:42:40.956826 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:40.956842 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:40.956941 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:40.957793 (+ 852us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:40.957999 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.959187 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.961774 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.962684 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.962855 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:40.962967 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:40.963265 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 19:42:40.963370 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12533,"thread_start_us":12981,"threads_started":1}
I20260807 19:42:40.963706 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.943331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:39325 (local address 127.0.0.1:49660)
0807 19:42:40.943498 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.943515 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:40.943626 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:40.957240 (+ 13614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:40.957246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:40.957278 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:40.957612 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:40.957626 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.959362 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.959368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:40.961546 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:40.961566 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:40.962924 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:40.962931 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:40.963102 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:40.963136 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:42:40.963250 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 19:42:40.963388 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 19:42:40.963507 7257 raft_consensus.cc:348] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:40.964525 7082 tablet_service.cc:1467] Processing CreateTablet for tablet 86677a320afa41ba8e34e68fd005f48f (DEFAULT_TABLE table=test-table [id=ca277f2a28094a7eb25ebc7e3301e69a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:40.964890 7082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86677a320afa41ba8e34e68fd005f48f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:40.965261 7257 raft_consensus.cc:374] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:40.965389 7257 raft_consensus.cc:729] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca911cece32a4d55ba20d1bb25ac8deb, State: Initialized, Role: FOLLOWER
I20260807 19:42:40.965610 7257 consensus_queue.cc:260] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:40.966499 7257 ts_tablet_manager.cc:1428] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 19:42:40.967167 7258 tablet_bootstrap.cc:492] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Bootstrap starting.
I20260807 19:42:40.969079 7258 tablet_bootstrap.cc:654] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:40.971625 6997 heartbeater.cc:503] Master 127.30.208.60:41979 was elected leader, sending a full tablet report...
I20260807 19:42:40.976739 7263 tablet_bootstrap.cc:492] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd: Bootstrap starting.
I20260807 19:42:40.976953 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.967098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.2:35789)
0807 19:42:40.976295 (+ 9197us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.976573 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9051,"thread_start_us":8897,"threads_started":1}
I20260807 19:42:40.978060 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.977728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.2:46177)
0807 19:42:40.977878 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.977955 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 19:42:40.978601 7258 tablet_bootstrap.cc:492] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: No bootstrap required, opened a new log
I20260807 19:42:40.978631 7263 tablet_bootstrap.cc:654] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:40.978777 7258 ts_tablet_manager.cc:1397] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.006s
I20260807 19:42:40.979681 7258 raft_consensus.cc:348] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:40.979959 7258 raft_consensus.cc:374] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:40.980058 7258 raft_consensus.cc:729] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a6ac8ef1cb744cbbf4d3916bf4d44ee, State: Initialized, Role: FOLLOWER
I20260807 19:42:40.980291 7258 consensus_queue.cc:260] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:40.981225 7263 tablet_bootstrap.cc:492] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd: No bootstrap required, opened a new log
I20260807 19:42:40.981400 7263 ts_tablet_manager.cc:1397] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 19:42:40.982263 7263 raft_consensus.cc:348] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:40.982506 7263 raft_consensus.cc:374] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:40.982605 7263 raft_consensus.cc:729] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 132385f9496d4ecbb98ec6d6419b46dd, State: Initialized, Role: FOLLOWER
I20260807 19:42:40.982849 7263 consensus_queue.cc:260] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:40.983657 7263 ts_tablet_manager.cc:1428] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 19:42:40.986022 7154 heartbeater.cc:503] Master 127.30.208.60:41979 was elected leader, sending a full tablet report...
I20260807 19:42:40.987380 7258 ts_tablet_manager.cc:1428] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Time spent starting tablet: real 0.008s user 0.004s sys 0.005s
I20260807 19:42:40.989353 6836 heartbeater.cc:503] Master 127.30.208.60:41979 was elected leader, sending a full tablet report...
I20260807 19:42:40.995812 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.987936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.1:35771)
0807 19:42:40.995423 (+ 7487us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.995577 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7337,"thread_start_us":7155,"threads_started":1}
I20260807 19:42:40.995905 7267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.984023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.3:39191)
0807 19:42:40.995553 (+ 11530us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.995681 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11414,"threads_started":1}
I20260807 19:42:40.996824 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.996447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.1:47591)
0807 19:42:40.996602 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.996690 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 19:42:40.997331 7267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:40.996759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.3:51437)
0807 19:42:40.997067 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:40.997140 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 19:42:41.074456 6839 tablet.cc:2369] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:41.289965 7260 raft_consensus.cc:482] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:41.290259 7260 raft_consensus.cc:504] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:41.292092 7260 leader_election.cc:288] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a6ac8ef1cb744cbbf4d3916bf4d44ee (127.30.208.1:35269), 132385f9496d4ecbb98ec6d6419b46dd (127.30.208.3:39325)
I20260807 19:42:41.298661 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.292492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:51529 (local address 127.30.208.1:35269)
0807 19:42:41.292702 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:41.292708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:41.292936 (+ 228us) server_negotiation.cc:407] Connection header received
0807 19:42:41.293160 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:41.293166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:41.293280 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:41.293361 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:41.293920 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.294946 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.296987 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.297789 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.297942 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:41.298018 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:41.298276 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 19:42:41.298415 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:42:41.298703 7261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.292353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:35269 (local address 127.30.208.2:51529)
0807 19:42:41.292764 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:41.292821 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:41.293014 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:41.293408 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:41.293413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:41.293461 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:41.293768 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:41.293788 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.295124 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.295130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:41.296828 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:41.296843 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.297958 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.297964 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:41.298122 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:41.298147 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:42:41.298239 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 19:42:41.298381 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":56,"spinlock_wait_cycles":768}
I20260807 19:42:41.299042 7269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.292550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:39325 (local address 127.30.208.2:56353)
0807 19:42:41.293326 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:41.293344 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:41.293473 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:41.293769 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:41.293773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:41.293799 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:41.294081 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:41.294090 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.295452 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.295458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:41.297097 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:41.297111 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.298246 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.298253 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:41.298402 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:41.298424 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 19:42:41.298501 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 19:42:41.298619 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":30,"spinlock_wait_cycles":27904,"thread_start_us":475,"threads_started":1}
I20260807 19:42:41.299053 7256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.292570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:56353 (local address 127.30.208.3:39325)
0807 19:42:41.292835 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:41.292842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:41.293434 (+ 592us) server_negotiation.cc:407] Connection header received
0807 19:42:41.293586 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:41.293589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:41.293649 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:41.293731 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:41.294236 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.295295 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.297260 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.298027 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.298182 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:41.298256 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:41.298544 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 19:42:41.298672 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260807 19:42:41.299574 6784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86677a320afa41ba8e34e68fd005f48f" candidate_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" is_pre_election: true
I20260807 19:42:41.299767 7105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86677a320afa41ba8e34e68fd005f48f" candidate_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "132385f9496d4ecbb98ec6d6419b46dd" is_pre_election: true
I20260807 19:42:41.300000 7105 raft_consensus.cc:2454] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca911cece32a4d55ba20d1bb25ac8deb in term 0.
I20260807 19:42:41.300010 6784 raft_consensus.cc:2454] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca911cece32a4d55ba20d1bb25ac8deb in term 0.
I20260807 19:42:41.300593 6878 leader_election.cc:302] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a6ac8ef1cb744cbbf4d3916bf4d44ee, ca911cece32a4d55ba20d1bb25ac8deb; no voters:
I20260807 19:42:41.301024 7260 raft_consensus.cc:2781] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:41.301139 7260 raft_consensus.cc:482] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:41.301259 7260 raft_consensus.cc:3037] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:41.303548 7260 raft_consensus.cc:504] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:41.304129 7260 leader_election.cc:288] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [CANDIDATE]: Term 1 election: Requested vote from peers 3a6ac8ef1cb744cbbf4d3916bf4d44ee (127.30.208.1:35269), 132385f9496d4ecbb98ec6d6419b46dd (127.30.208.3:39325)
I20260807 19:42:41.304587 6784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86677a320afa41ba8e34e68fd005f48f" candidate_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee"
I20260807 19:42:41.304632 7105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86677a320afa41ba8e34e68fd005f48f" candidate_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "132385f9496d4ecbb98ec6d6419b46dd"
I20260807 19:42:41.304776 6784 raft_consensus.cc:3037] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:41.304822 7105 raft_consensus.cc:3037] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:41.306849 7105 raft_consensus.cc:2454] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca911cece32a4d55ba20d1bb25ac8deb in term 1.
I20260807 19:42:41.307219 6784 raft_consensus.cc:2454] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca911cece32a4d55ba20d1bb25ac8deb in term 1.
I20260807 19:42:41.307315 6877 leader_election.cc:302] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 132385f9496d4ecbb98ec6d6419b46dd, ca911cece32a4d55ba20d1bb25ac8deb; no voters:
I20260807 19:42:41.307639 7260 raft_consensus.cc:2781] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:41.308382 7260 raft_consensus.cc:686] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [term 1 LEADER]: Becoming Leader. State: Replica: ca911cece32a4d55ba20d1bb25ac8deb, State: Running, Role: LEADER
I20260807 19:42:41.308799 7260 consensus_queue.cc:237] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } }
I20260807 19:42:41.309435 7269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.308945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.2:41697)
0807 19:42:41.309128 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:41.309255 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 19:42:41.312264 6636 catalog_manager.cc:5495] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb reported cstate change: term changed from 0 to 1, leader changed from <none> to ca911cece32a4d55ba20d1bb25ac8deb (127.30.208.2). New cstate: current_term: 1 leader_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca911cece32a4d55ba20d1bb25ac8deb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38995 } health_report { overall_health: HEALTHY } } }
I20260807 19:42:41.393558 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.393063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:54108)
0807 19:42:41.393309 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:41.393406 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 19:42:41.394572 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.394201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:54116)
0807 19:42:41.394344 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:41.394429 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 19:42:41.407193 7255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.402353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52968 (local address 127.30.208.2:38995)
0807 19:42:41.402531 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:41.402536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:41.402553 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:42:41.402683 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:41.402687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:41.402774 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:41.402899 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:41.403247 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.403947 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.405284 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.405716 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.405808 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:41.406184 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:41.406733 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:41.406833 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:41.406926 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 19:42:41.406990 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:42:41.411006 7105 raft_consensus.cc:1261] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd [term 1 FOLLOWER]: Refusing update from remote peer ca911cece32a4d55ba20d1bb25ac8deb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:41.411438 6784 raft_consensus.cc:1261] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 1 FOLLOWER]: Refusing update from remote peer ca911cece32a4d55ba20d1bb25ac8deb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:41.411679 7270 consensus_queue.cc:1035] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "132385f9496d4ecbb98ec6d6419b46dd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:41.412091 7260 consensus_queue.cc:1035] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a6ac8ef1cb744cbbf4d3916bf4d44ee" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:41.414583 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.413932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.1:36143)
0807 19:42:41.414219 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:41.414343 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260807 19:42:41.415416 7267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.414774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.3:43437)
0807 19:42:41.415122 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:41.415219 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236}
I20260807 19:42:41.420913 7267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.420508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.3:38567)
0807 19:42:41.420654 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:41.420742 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 19:42:41.421902 7269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.421045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.2:59181)
0807 19:42:41.421674 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:41.421774 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519}
I20260807 19:42:41.425537 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.424785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.30.208.1:33045)
0807 19:42:41.425294 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:41.425386 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373}
I20260807 19:42:41.437700 6635 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:41.438002 6635 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:41.444244 6635 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61adb8b777d7419196442d1345e0dd55: Sending DeleteTablet for 3 replicas of tablet 86677a320afa41ba8e34e68fd005f48f
I20260807 19:42:41.445312 6761 tablet_service.cc:1514] Processing DeleteTablet for tablet 86677a320afa41ba8e34e68fd005f48f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:41 UTC) from {username='oryx', principal='oryx/127.30.208.60@KRBTEST.COM'} at 127.0.0.1:51644
I20260807 19:42:41.445505 7082 tablet_service.cc:1514] Processing DeleteTablet for tablet 86677a320afa41ba8e34e68fd005f48f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:41 UTC) from {username='oryx', principal='oryx/127.30.208.60@KRBTEST.COM'} at 127.0.0.1:49660
I20260807 19:42:41.446081 7285 tablet_replica.cc:330] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: stopping tablet replica
I20260807 19:42:41.446259 6926 tablet_service.cc:1514] Processing DeleteTablet for tablet 86677a320afa41ba8e34e68fd005f48f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:41 UTC) from {username='oryx', principal='oryx/127.30.208.60@KRBTEST.COM'} at 127.0.0.1:52952
I20260807 19:42:41.446483 7285 raft_consensus.cc:2229] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:41.446830 7285 raft_consensus.cc:2258] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:41.448109 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 6624
I20260807 19:42:41.448097 7285 ts_tablet_manager.cc:1905] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:41.455281 7285 ts_tablet_manager.cc:1918] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:41.455549 7285 log.cc:1196] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/86677a320afa41ba8e34e68fd005f48f
I20260807 19:42:41.456132 7285 ts_tablet_manager.cc:1939] T 86677a320afa41ba8e34e68fd005f48f P 3a6ac8ef1cb744cbbf4d3916bf4d44ee: Deleting consensus metadata
I20260807 19:42:41.457449 6619 catalog_manager.cc:4841] TS 3a6ac8ef1cb744cbbf4d3916bf4d44ee (127.30.208.1:35269): tablet 86677a320afa41ba8e34e68fd005f48f (table test-table [id=ca277f2a28094a7eb25ebc7e3301e69a]) successfully deleted
I20260807 19:42:41.461313 7287 tablet_replica.cc:330] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb: stopping tablet replica
I20260807 19:42:41.461513 7287 raft_consensus.cc:2229] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:41.462059 7287 raft_consensus.cc:2258] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:41.462953 7287 ts_tablet_manager.cc:1905] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:41.466382 7286 tablet_replica.cc:330] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd: stopping tablet replica
I20260807 19:42:41.466586 7286 raft_consensus.cc:2229] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:41.466897 7286 raft_consensus.cc:2258] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:41.468299 7286 ts_tablet_manager.cc:1905] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:41.471606 7287 ts_tablet_manager.cc:1918] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:41.471833 7287 log.cc:1196] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/86677a320afa41ba8e34e68fd005f48f
I20260807 19:42:41.472415 7287 ts_tablet_manager.cc:1939] T 86677a320afa41ba8e34e68fd005f48f P ca911cece32a4d55ba20d1bb25ac8deb: Deleting consensus metadata
I20260807 19:42:41.473908 6618 catalog_manager.cc:4841] TS ca911cece32a4d55ba20d1bb25ac8deb (127.30.208.2:38995): tablet 86677a320afa41ba8e34e68fd005f48f (table test-table [id=ca277f2a28094a7eb25ebc7e3301e69a]) successfully deleted
I20260807 19:42:41.474732 7286 ts_tablet_manager.cc:1918] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:41.474869 7286 log.cc:1196] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/86677a320afa41ba8e34e68fd005f48f
I20260807 19:42:41.475368 7286 ts_tablet_manager.cc:1939] T 86677a320afa41ba8e34e68fd005f48f P 132385f9496d4ecbb98ec6d6419b46dd: Deleting consensus metadata
I20260807 19:42:41.476608 6618 catalog_manager.cc:4841] TS 132385f9496d4ecbb98ec6d6419b46dd (127.30.208.3:39325): tablet 86677a320afa41ba8e34e68fd005f48f (table test-table [id=ca277f2a28094a7eb25ebc7e3301e69a]) successfully deleted
I20260807 19:42:41.483158 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.449224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51660 (local address 127.30.208.1:35269)
0807 19:42:41.449387 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:41.449392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:41.465794 (+ 16402us) server_negotiation.cc:407] Connection header received
0807 19:42:41.465990 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:41.465995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:41.466083 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:41.466175 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:41.468576 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.469775 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.471531 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.472100 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.472197 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:41.476718 (+ 4521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:41.476741 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:41.476762 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:41.476815 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:41.480067 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:41.481011 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:41.481018 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:41.481020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:41.481168 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:41.481657 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:41.481661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:41.481663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:41.482067 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:41.482300 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:41.482649 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 19:42:41.482873 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 19:42:41.483851 6830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:41.636780 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 6624
I20260807 19:42:41.665084 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 6851
I20260807 19:42:41.681811 7255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.665894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52972 (local address 127.30.208.2:38995)
0807 19:42:41.666514 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:41.666520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:41.666542 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:42:41.666611 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:41.666616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:41.666733 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:41.666842 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:41.668412 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.669560 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.670820 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.671499 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.671643 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:41.674333 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:41.674350 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:41.674372 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:41.674415 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:41.678643 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:41.679469 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:41.679478 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:41.679482 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:41.679719 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:41.680108 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:41.680113 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:41.680116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:41.680621 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:41.680990 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:41.681156 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 19:42:41.681416 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486}
I20260807 19:42:41.682574 6992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:41.804152 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 6851
I20260807 19:42:41.824956 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 7014
I20260807 19:42:41.839928 7291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.825741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49674 (local address 127.30.208.3:39325)
0807 19:42:41.826229 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:41.826233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:41.826249 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:42:41.826303 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:41.826306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:41.826380 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:41.826475 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:41.827900 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.828632 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.829934 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:41.830543 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:41.830689 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:41.833474 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:41.833491 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:41.833507 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:41.833545 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:41.836780 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:41.837814 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:41.837823 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:41.837825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:41.837957 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:41.838343 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:41.838347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:41.838349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:41.838733 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:41.839036 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:41.839362 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 19:42:41.839604 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":255,"threads_started":1}
I20260807 19:42:41.840657 7148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:41.934437 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 7014
I20260807 19:42:41.943796 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.942989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:54130)
0807 19:42:41.943483 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:41.943582 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":249,"threads_started":1}
I20260807 19:42:41.945079 7293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:41.944708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33613 (local address 127.0.0.1:54142)
0807 19:42:41.944873 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:41.944954 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 19:42:41.953907 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 6279
I20260807 19:42:41.954962 6465 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:42.073905 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 6279
I20260807 19:42:42.095793 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 6395
I20260807 19:42:42.096890 6669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:42.221005 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 6395
2026-08-07T19:42:42Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9809 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:42 dist-test-slave-vq2q krb5kdc[7299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:42 dist-test-slave-vq2q krb5kdc[7299](info): set up 2 sockets
Aug 07 19:42:42 dist-test-slave-vq2q krb5kdc[7299](info): commencing operation
krb5kdc: starting...
W20260807 19:42:43.308910 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:42:43 dist-test-slave-vq2q krb5kdc[7299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:43Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.62.keytab.
Entry for principal oryx/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.62.keytab.
I20260807 19:42:43.557515 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:45093
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.62.keytab
--principal=oryx/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:37429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:45093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:43.894145 7313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:43.894443 7313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:43.894560 7313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:43.901933 7313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:43.902048 7313 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 19:42:43.902092 7313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:43.902179 7313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:43.902237 7313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:43.902323 7313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:43.912377 7313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:45093
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.62.keytab
--principal=oryx/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:45093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:43.913764 7313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:43.915133 7313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:43.928004 7321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:43.928081 7313 server_base.cc:1034] running on GCE node
W20260807 19:42:43.928061 7319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:43.927914 7318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:43.929384 7313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:43.930869 7313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:43.932093 7313 hybrid_clock.cc:648] HybridClock initialized: now 1786131763932027 us; error 38 us; skew 500 ppm
I20260807 19:42:43.932442 7313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:43 dist-test-slave-vq2q krb5kdc[7299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131763, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:43.935627 7313 init.cc:377] Logged in from keytab as oryx/127.30.208.62@KRBTEST.COM (short username oryx)
I20260807 19:42:43.937127 7313 webserver.cc:460] Webserver started at http://127.30.208.62:38415/ using document root <none> and password file <none>
I20260807 19:42:43.937877 7313 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:43.937973 7313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:43.938302 7313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:43.941469 7313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "f32b18cc46444585af278750d8ceef8f"
format_stamp: "Formatted at 2026-08-07 19:42:43 on dist-test-slave-vq2q"
server_key: "ebf91c83666f7a528256e13f7009b498"
server_key_iv: "1ddec07b235154d82e6685f12a055d96"
server_key_version: "encryptionkey@0"
I20260807 19:42:43.942328 7313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "f32b18cc46444585af278750d8ceef8f"
format_stamp: "Formatted at 2026-08-07 19:42:43 on dist-test-slave-vq2q"
server_key: "ebf91c83666f7a528256e13f7009b498"
server_key_iv: "1ddec07b235154d82e6685f12a055d96"
server_key_version: "encryptionkey@0"
I20260807 19:42:43.947616 7313 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:42:43.951299 7329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:43.952739 7313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:42:43.952906 7313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "f32b18cc46444585af278750d8ceef8f"
format_stamp: "Formatted at 2026-08-07 19:42:43 on dist-test-slave-vq2q"
server_key: "ebf91c83666f7a528256e13f7009b498"
server_key_iv: "1ddec07b235154d82e6685f12a055d96"
server_key_version: "encryptionkey@0"
I20260807 19:42:43.953120 7313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:43.979153 7313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:43.983621 7313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:43.983896 7313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:44.006830 7313 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:45093
I20260807 19:42:44.006881 7400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:45093 every 8 connection(s)
I20260807 19:42:44.008409 7313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:44.012320 7401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:44.016162 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 7313
I20260807 19:42:44.016436 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:42:44.017272 31552 external_mini_cluster.cc:1428] Setting key c1d336a94c455078a87ccb155a239eb2
I20260807 19:42:44.026242 7401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f: Bootstrap starting.
I20260807 19:42:44.030099 7401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:44.031417 7401 log.cc:824] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:42:44 dist-test-slave-vq2q krb5kdc[7299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
I20260807 19:42:44.034732 7401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f: No bootstrap required, opened a new log
I20260807 19:42:44.042289 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:44.019497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38670 (local address 127.30.208.62:45093)
0807 19:42:44.020308 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:44.020321 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:44.020365 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:42:44.022114 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:44.022146 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:44.022586 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:44.022946 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:44.024731 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:44.026094 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:44.027850 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:44.028613 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:44.028756 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:44.033537 (+ 4781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:44.033549 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:44.033568 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:44.033620 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:44.037571 (+ 3951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:44.038242 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:44.038251 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:44.038260 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:44.038479 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:44.038898 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:44.038903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:44.038906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:44.039633 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:44.039952 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:44.040340 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 19:42:44.040819 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":372,"threads_started":1}
I20260807 19:42:44.044979 7401 raft_consensus.cc:348] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f32b18cc46444585af278750d8ceef8f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45093 } }
I20260807 19:42:44.045470 7401 raft_consensus.cc:374] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:44.045572 7401 raft_consensus.cc:729] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f32b18cc46444585af278750d8ceef8f, State: Initialized, Role: FOLLOWER
I20260807 19:42:44.046120 7401 consensus_queue.cc:260] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f32b18cc46444585af278750d8ceef8f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45093 } }
I20260807 19:42:44.046340 7401 raft_consensus.cc:388] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:44.046469 7401 raft_consensus.cc:482] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:44.046610 7401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:44.048918 7401 raft_consensus.cc:504] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f32b18cc46444585af278750d8ceef8f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45093 } }
I20260807 19:42:44.049440 7401 leader_election.cc:302] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f32b18cc46444585af278750d8ceef8f; no voters:
I20260807 19:42:44.050077 7401 leader_election.cc:288] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:44.050213 7407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:44.051198 7407 raft_consensus.cc:686] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [term 1 LEADER]: Becoming Leader. State: Replica: f32b18cc46444585af278750d8ceef8f, State: Running, Role: LEADER
I20260807 19:42:44.051712 7407 consensus_queue.cc:237] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f32b18cc46444585af278750d8ceef8f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45093 } }
I20260807 19:42:44.052021 7401 sys_catalog.cc:564] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:44.056041 7409 sys_catalog.cc:455] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f32b18cc46444585af278750d8ceef8f. Latest consensus state: current_term: 1 leader_uuid: "f32b18cc46444585af278750d8ceef8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f32b18cc46444585af278750d8ceef8f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45093 } } }
I20260807 19:42:44.056079 7408 sys_catalog.cc:455] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f32b18cc46444585af278750d8ceef8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f32b18cc46444585af278750d8ceef8f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45093 } } }
I20260807 19:42:44.056452 7408 sys_catalog.cc:458] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:44.056449 7409 sys_catalog.cc:458] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:44.058176 7418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:44.063205 7418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:44.072615 7418 catalog_manager.cc:1347] Generated new cluster ID: a4e324c2b9414851b54ffe4c19a4c4e5
I20260807 19:42:44.072736 7418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:44.103157 7418 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:44.104485 7418 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:44.110659 7418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f: Generated new TSK 0
I20260807 19:42:44.111680 7418 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.1.keytab.
Entry for principal oryx/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.1.keytab.
I20260807 19:42:44.176666 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45093
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.1.keytab
--principal=oryx/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:37429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:44.516106 7430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:44.516407 7430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:44.516507 7430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:44.524037 7430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:44.524147 7430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:44.524236 7430 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:42:44.524317 7430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:42:44.534511 7430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.1.keytab
--principal=oryx/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.208.62:45093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:44.535801 7430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:44.537205 7430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:44.550081 7438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:44.550163 7436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:44.550143 7435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:44.551303 7430 server_base.cc:1034] running on GCE node
I20260807 19:42:44.551970 7430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:44.552989 7430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:44.554207 7430 hybrid_clock.cc:648] HybridClock initialized: now 1786131764554129 us; error 50 us; skew 500 ppm
I20260807 19:42:44.554492 7430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:44 dist-test-slave-vq2q krb5kdc[7299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:44.557808 7430 init.cc:377] Logged in from keytab as oryx/127.30.208.1@KRBTEST.COM (short username oryx)
I20260807 19:42:44.559273 7430 webserver.cc:460] Webserver started at http://127.30.208.1:36767/ using document root <none> and password file <none>
I20260807 19:42:44.559984 7430 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:44.560088 7430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:44.560415 7430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:44.563486 7430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "92b8396827f24c6887da469b9b816895"
format_stamp: "Formatted at 2026-08-07 19:42:44 on dist-test-slave-vq2q"
server_key: "afd2103dc43cb0c436632b8d141a611d"
server_key_iv: "0f309ba55ddbbcacdee8b35064582348"
server_key_version: "encryptionkey@0"
I20260807 19:42:44.564282 7430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "92b8396827f24c6887da469b9b816895"
format_stamp: "Formatted at 2026-08-07 19:42:44 on dist-test-slave-vq2q"
server_key: "afd2103dc43cb0c436632b8d141a611d"
server_key_iv: "0f309ba55ddbbcacdee8b35064582348"
server_key_version: "encryptionkey@0"
I20260807 19:42:44.569574 7430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:42:44.573225 7446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:44.574553 7430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:44.574712 7430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "92b8396827f24c6887da469b9b816895"
format_stamp: "Formatted at 2026-08-07 19:42:44 on dist-test-slave-vq2q"
server_key: "afd2103dc43cb0c436632b8d141a611d"
server_key_iv: "0f309ba55ddbbcacdee8b35064582348"
server_key_version: "encryptionkey@0"
I20260807 19:42:44.574934 7430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:44.592779 7430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:44.596732 7430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:44.596989 7430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:44.599691 7430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:44.599778 7430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:44.599926 7430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:44.599984 7430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:42:44 dist-test-slave-vq2q krb5kdc[7299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131764, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
I20260807 19:42:44.623571 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:44.600403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:35033 (local address 127.30.208.62:45093)
0807 19:42:44.600916 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:44.600924 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:44.602102 (+ 1178us) server_negotiation.cc:407] Connection header received
0807 19:42:44.604195 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:44.604201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:44.604296 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:44.604407 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:44.607250 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:44.608207 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:44.610744 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:44.611310 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:44.611444 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:44.616840 (+ 5396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:44.616845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:44.616849 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:44.616886 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:44.619626 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:44.620652 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:44.620656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:44.620658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:44.620783 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:44.621425 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:44.621430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:44.621433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:44.621773 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:44.622061 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:44.623085 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0807 19:42:44.623318 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":250,"threads_started":1}
I20260807 19:42:44.625543 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:44.601013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:45093 (local address 127.30.208.1:35033)
0807 19:42:44.601912 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:44.601959 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:44.603888 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:44.604874 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:44.604891 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:44.605937 (+ 1046us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:44.606997 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:44.607034 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:44.608410 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:44.608414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:44.610562 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:44.610580 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:44.611625 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:44.611636 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:44.612087 (+ 451us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:44.612927 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:44.612962 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:44.616509 (+ 3547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:44.619797 (+ 3288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:44.619808 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:44.619830 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:44.620527 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:44.620922 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:44.620927 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:44.620932 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:44.621269 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:44.621951 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:44.621961 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:44.622567 (+ 606us) client_negotiation.cc:769] Sending connection context
0807 19:42:44.622965 (+ 398us) client_negotiation.cc:240] Negotiation successful
0807 19:42:44.623624 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":402,"threads_started":1}
I20260807 19:42:44.646852 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:44.631655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:49835 (local address 127.30.208.62:45093)
0807 19:42:44.631808 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:44.631812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:44.632486 (+ 674us) server_negotiation.cc:407] Connection header received
0807 19:42:44.632682 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:44.632688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:44.632777 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:44.632884 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:44.634484 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:44.635242 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:44.637482 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:44.637999 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:44.638149 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:44.640809 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:44.640817 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:44.640822 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:44.640860 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:44.644018 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:44.644747 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:44.644753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:44.644756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:44.644911 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:44.645291 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:44.645296 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:44.645299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:44.645648 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:44.645833 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:44.646410 (+ 577us) server_negotiation.cc:299] Negotiation successful
0807 19:42:44.646609 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 19:42:44.647185 7540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:44.631723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:45093 (local address 127.30.208.1:49835)
0807 19:42:44.632379 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:44.632396 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:44.632547 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:44.632953 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:44.632961 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:44.633740 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:44.634290 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:44.634307 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:44.635446 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:44.635453 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:44.637268 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:44.637290 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:44.638232 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:44.638239 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:44.638418 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:44.639116 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:44.639144 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:44.640535 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:44.644178 (+ 3643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:44.644184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:44.644189 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:44.644622 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:44.645024 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:44.645029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:44.645033 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:44.645168 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:44.645805 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:44.645814 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:44.646057 (+ 243us) client_negotiation.cc:769] Sending connection context
0807 19:42:44.646229 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 19:42:44.646652 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":373,"threads_started":1}
I20260807 19:42:44.649932 7430 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:37733
I20260807 19:42:44.649973 7575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:37733 every 8 connection(s)
I20260807 19:42:44.651446 7430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:44.652882 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 7430
I20260807 19:42:44.653090 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:44.653388 7576 heartbeater.cc:347] Connected to a master server at 127.30.208.62:45093
I20260807 19:42:44.653612 7576 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:44.653842 31552 external_mini_cluster.cc:1428] Setting key 85f83a17ee169aee1c4901a73e304b37
I20260807 19:42:44.654325 7576 heartbeater.cc:511] Master 127.30.208.62:45093 requested a full tablet report, sending...
I20260807 19:42:44.656391 7349 ts_manager.cc:194] Registered new tserver with Master: 92b8396827f24c6887da469b9b816895 (127.30.208.1:37733)
I20260807 19:42:44.657966 7349 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:35033
WARNING: no policy specified for oryx/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.2.keytab.
Entry for principal oryx/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.2.keytab.
I20260807 19:42:44.708207 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45093
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.2.keytab
--principal=oryx/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:37429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:45.045271 7580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:45.045591 7580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:45.045761 7580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:45.053333 7580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:45.053474 7580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:45.053601 7580 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:42:45.053721 7580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:42:45.064131 7580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.2.keytab
--principal=oryx/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.208.62:45093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:45.065470 7580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:45.066848 7580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:45.080019 7589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:45.080077 7588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:45.080057 7591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:45.080845 7580 server_base.cc:1034] running on GCE node
I20260807 19:42:45.081549 7580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:45.082494 7580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:45.083705 7580 hybrid_clock.cc:648] HybridClock initialized: now 1786131765083637 us; error 41 us; skew 500 ppm
I20260807 19:42:45.084008 7580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:45 dist-test-slave-vq2q krb5kdc[7299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131765, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:45.087286 7580 init.cc:377] Logged in from keytab as oryx/127.30.208.2@KRBTEST.COM (short username oryx)
I20260807 19:42:45.088773 7580 webserver.cc:460] Webserver started at http://127.30.208.2:40115/ using document root <none> and password file <none>
I20260807 19:42:45.089509 7580 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:45.089603 7580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:45.089919 7580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:45.092832 7580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "9d6b390dce8b4ab5b978ea2286e63554"
format_stamp: "Formatted at 2026-08-07 19:42:45 on dist-test-slave-vq2q"
server_key: "38b85e654a03d2fe9a668110f02727da"
server_key_iv: "5e93a90cbc4068f350c4ddc2b35c3fe5"
server_key_version: "encryptionkey@0"
I20260807 19:42:45.093681 7580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "9d6b390dce8b4ab5b978ea2286e63554"
format_stamp: "Formatted at 2026-08-07 19:42:45 on dist-test-slave-vq2q"
server_key: "38b85e654a03d2fe9a668110f02727da"
server_key_iv: "5e93a90cbc4068f350c4ddc2b35c3fe5"
server_key_version: "encryptionkey@0"
I20260807 19:42:45.098717 7580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:42:45.102084 7598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:45.103430 7580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:45.103581 7580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "9d6b390dce8b4ab5b978ea2286e63554"
format_stamp: "Formatted at 2026-08-07 19:42:45 on dist-test-slave-vq2q"
server_key: "38b85e654a03d2fe9a668110f02727da"
server_key_iv: "5e93a90cbc4068f350c4ddc2b35c3fe5"
server_key_version: "encryptionkey@0"
I20260807 19:42:45.103792 7580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:45.121563 7580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:45.125841 7580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:45.126094 7580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:45.128793 7580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:45.128876 7580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:45.129014 7580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:45.129069 7580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:42:45 dist-test-slave-vq2q krb5kdc[7299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131765, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
I20260807 19:42:45.153007 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:45.129484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:34457 (local address 127.30.208.62:45093)
0807 19:42:45.129679 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:45.129685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:45.131139 (+ 1454us) server_negotiation.cc:407] Connection header received
0807 19:42:45.134160 (+ 3021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:45.134167 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:45.134268 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:45.134386 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:45.136591 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.137600 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.140540 (+ 2940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.141138 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.141312 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:45.145357 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:45.145364 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:45.145369 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:45.145415 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:45.148927 (+ 3512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:45.149883 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:45.149887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:45.149889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:45.150015 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:45.150687 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:45.150692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:45.150695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:45.151102 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:45.151269 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:45.152280 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0807 19:42:45.152564 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:42:45.153874 7607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:45.130048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:45093 (local address 127.30.208.2:34457)
0807 19:42:45.130939 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:45.131009 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:45.133767 (+ 2758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:45.134678 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:45.134687 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:45.135467 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:45.136396 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:45.136426 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.137783 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.137789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:45.140363 (+ 2574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:45.140382 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.141426 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.141435 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:45.141674 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:45.142353 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:45.142377 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:45.145061 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:45.149073 (+ 4012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:45.149083 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:45.149100 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:45.149712 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:45.150133 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:45.150139 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:45.150142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:45.150492 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:45.151261 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:45.151270 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:45.151756 (+ 486us) client_negotiation.cc:769] Sending connection context
0807 19:42:45.152083 (+ 327us) client_negotiation.cc:240] Negotiation successful
0807 19:42:45.152580 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":338,"threads_started":1}
I20260807 19:42:45.174768 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:45.159934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:47043 (local address 127.30.208.62:45093)
0807 19:42:45.160096 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:45.160099 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:45.160701 (+ 602us) server_negotiation.cc:407] Connection header received
0807 19:42:45.160843 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:45.160849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:45.160937 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:45.161047 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:45.162251 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.163162 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.165289 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.165852 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.165979 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:45.168226 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:45.168231 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:45.168234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:45.168274 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:45.171630 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:45.172403 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:45.172408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:45.172411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:45.172582 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:45.172946 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:45.172950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:45.172954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:45.173507 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:45.173679 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:45.174180 (+ 501us) server_negotiation.cc:299] Negotiation successful
0807 19:42:45.174407 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:42:45.174865 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:45.160040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:45093 (local address 127.30.208.2:47043)
0807 19:42:45.160604 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:45.160619 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:45.160735 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:45.161058 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:45.161061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:45.161639 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:45.162099 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:45.162111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.163337 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.163366 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:45.165084 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:45.165102 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.166081 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.166089 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:45.166290 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:45.166905 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:45.166928 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:45.168043 (+ 1115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:45.171793 (+ 3750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:45.171803 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:45.171806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:45.172275 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:45.172700 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:45.172703 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:45.172706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:45.172844 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:45.173715 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:45.173720 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:45.173924 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 19:42:45.174064 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 19:42:45.174415 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":305,"threads_started":1}
I20260807 19:42:45.183947 7580 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:38301
I20260807 19:42:45.184015 7730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:38301 every 8 connection(s)
I20260807 19:42:45.185534 7580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:45.187738 7731 heartbeater.cc:347] Connected to a master server at 127.30.208.62:45093
I20260807 19:42:45.187937 7731 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:45.188541 7731 heartbeater.cc:511] Master 127.30.208.62:45093 requested a full tablet report, sending...
I20260807 19:42:45.189824 7349 ts_manager.cc:194] Registered new tserver with Master: 9d6b390dce8b4ab5b978ea2286e63554 (127.30.208.2:38301)
I20260807 19:42:45.190623 7349 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:34457
I20260807 19:42:45.194216 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 7580
I20260807 19:42:45.194423 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:45.194824 31552 external_mini_cluster.cc:1428] Setting key 1292744f6029f8d4b04cab3ada0d0df0
WARNING: no policy specified for oryx/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.3.keytab.
Entry for principal oryx/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.3.keytab.
I20260807 19:42:45.248693 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45093
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.3.keytab
--principal=oryx/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:37429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:45.582784 7737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:45.583081 7737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:45.583230 7737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:45.590514 7737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:45.590646 7737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:45.590773 7737 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 19:42:45.590886 7737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:42:45.600828 7737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.3.keytab
--principal=oryx/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.208.62:45093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:45.602134 7737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:45.603458 7737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:45.616396 7742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:45.616472 7745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:45.616425 7743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:45.617208 7737 server_base.cc:1034] running on GCE node
I20260807 19:42:45.617857 7737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:45.618804 7737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:45.620013 7737 hybrid_clock.cc:648] HybridClock initialized: now 1786131765619943 us; error 46 us; skew 500 ppm
I20260807 19:42:45.620294 7737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:45 dist-test-slave-vq2q krb5kdc[7299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131765, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:45.623368 7737 init.cc:377] Logged in from keytab as oryx/127.30.208.3@KRBTEST.COM (short username oryx)
I20260807 19:42:45.624754 7737 webserver.cc:460] Webserver started at http://127.30.208.3:35717/ using document root <none> and password file <none>
I20260807 19:42:45.625452 7737 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:45.625540 7737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:45.625833 7737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:45.628839 7737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "2677256de76840bfbd379bcb63c4ea52"
format_stamp: "Formatted at 2026-08-07 19:42:45 on dist-test-slave-vq2q"
server_key: "937519481fabc36f74e93bb28c875b54"
server_key_iv: "28fc52a4ce3cbb52064b3753da2298bf"
server_key_version: "encryptionkey@0"
I20260807 19:42:45.629629 7737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "2677256de76840bfbd379bcb63c4ea52"
format_stamp: "Formatted at 2026-08-07 19:42:45 on dist-test-slave-vq2q"
server_key: "937519481fabc36f74e93bb28c875b54"
server_key_iv: "28fc52a4ce3cbb52064b3753da2298bf"
server_key_version: "encryptionkey@0"
I20260807 19:42:45.634487 7737 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 19:42:45.638059 7754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:45.639431 7737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:45.639595 7737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "2677256de76840bfbd379bcb63c4ea52"
format_stamp: "Formatted at 2026-08-07 19:42:45 on dist-test-slave-vq2q"
server_key: "937519481fabc36f74e93bb28c875b54"
server_key_iv: "28fc52a4ce3cbb52064b3753da2298bf"
server_key_version: "encryptionkey@0"
I20260807 19:42:45.639807 7737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:45.661085 7576 heartbeater.cc:503] Master 127.30.208.62:45093 was elected leader, sending a full tablet report...
I20260807 19:42:45.664573 7737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:45.668582 7737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:45.668830 7737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:45.671451 7737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:45.671541 7737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:45.671679 7737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:45.671742 7737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:42:45 dist-test-slave-vq2q krb5kdc[7299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131765, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
I20260807 19:42:45.694947 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:45.672044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:55915 (local address 127.30.208.62:45093)
0807 19:42:45.672230 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:45.672237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:45.673697 (+ 1460us) server_negotiation.cc:407] Connection header received
0807 19:42:45.676178 (+ 2481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:45.676184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:45.676288 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:45.676377 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:45.678715 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.679723 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.681788 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.682322 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.682446 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:45.687026 (+ 4580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:45.687031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:45.687034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:45.687072 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:45.690751 (+ 3679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:45.691866 (+ 1115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:45.691872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:45.691874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:45.692046 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:45.692819 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:45.692824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:45.692828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:45.693273 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:45.693445 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:45.694259 (+ 814us) server_negotiation.cc:299] Negotiation successful
0807 19:42:45.694541 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:42:45.695755 7765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:45.672622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:45093 (local address 127.30.208.3:55915)
0807 19:42:45.673530 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:45.673575 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:45.675855 (+ 2280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:45.676675 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:45.676687 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:45.677523 (+ 836us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:45.678458 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:45.678490 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.679921 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.679930 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:45.681549 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:45.681569 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.682706 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.682711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:45.682954 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:45.683697 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:45.683729 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:45.686713 (+ 2984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:45.690940 (+ 4227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:45.690954 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:45.690974 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:45.691712 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:45.692242 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:45.692248 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:45.692267 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:45.692647 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:45.693426 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:45.693435 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:45.693820 (+ 385us) client_negotiation.cc:769] Sending connection context
0807 19:42:45.694064 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 19:42:45.694475 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":337,"threads_started":1}
I20260807 19:42:45.717782 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:45.702587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:33209 (local address 127.30.208.62:45093)
0807 19:42:45.702795 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:45.702801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:45.703512 (+ 711us) server_negotiation.cc:407] Connection header received
0807 19:42:45.703745 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:45.703750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:45.703827 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:45.703915 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:45.705157 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.706102 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.708375 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.708959 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.709067 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:45.711601 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:45.711607 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:45.711610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:45.711646 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:45.714362 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:45.715280 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:45.715294 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:45.715297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:45.715451 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:45.715951 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:45.715955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:45.715957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:45.716314 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:45.716465 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:45.717087 (+ 622us) server_negotiation.cc:299] Negotiation successful
0807 19:42:45.717403 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:42:45.717809 7851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:45.702602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:45093 (local address 127.30.208.3:33209)
0807 19:42:45.703389 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:45.703411 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:45.703579 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:45.703957 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:45.703961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:45.704587 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:45.704988 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:45.705002 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.706270 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.706276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:45.708188 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:45.708207 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.709211 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.709219 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:45.709440 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:45.710162 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:45.710187 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:45.711377 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:45.714503 (+ 3126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:45.714509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:45.714513 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:45.715136 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:45.715575 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:45.715578 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:45.715581 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:45.715767 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:45.716500 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:45.716505 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:45.716730 (+ 225us) client_negotiation.cc:769] Sending connection context
0807 19:42:45.716950 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 19:42:45.717349 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":452,"threads_started":1}
I20260807 19:42:45.723145 7737 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:43607
I20260807 19:42:45.723205 7887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:43607 every 8 connection(s)
I20260807 19:42:45.724424 7737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:45.726483 7888 heartbeater.cc:347] Connected to a master server at 127.30.208.62:45093
I20260807 19:42:45.726698 7888 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:45.727362 7888 heartbeater.cc:511] Master 127.30.208.62:45093 requested a full tablet report, sending...
I20260807 19:42:45.728714 7349 ts_manager.cc:194] Registered new tserver with Master: 2677256de76840bfbd379bcb63c4ea52 (127.30.208.3:43607)
I20260807 19:42:45.729609 7349 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:55915
I20260807 19:42:45.733708 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 7737
I20260807 19:42:45.733872 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:45.734258 31552 external_mini_cluster.cc:1428] Setting key b95f33623581e9455ec31198a6ad717e
I20260807 19:42:45.737561 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 19:42:45 dist-test-slave-vq2q krb5kdc[7299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131765, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 19:42:45 dist-test-slave-vq2q krb5kdc[7299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:45 dist-test-slave-vq2q krb5kdc[7299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:42:46.192979 7731 heartbeater.cc:503] Master 127.30.208.62:45093 was elected leader, sending a full tablet report...
I20260807 19:42:46.731961 7888 heartbeater.cc:503] Master 127.30.208.62:45093 was elected leader, sending a full tablet report...
W20260807 19:42:46.829246 7897 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.039s user 0.004s sys 0.013s
W20260807 19:42:46.830452 7455 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:45.784739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.30.208.62:45093)
0807 19:42:45.784994 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:45.785002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:45.785199 (+ 197us) server_negotiation.cc:407] Connection header received
0807 19:42:45.785409 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:45.785413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:45.785496 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:45.785578 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:45.786876 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.787610 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.788789 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:45.789234 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:45.789346 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:46.830169 (+1040823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38678: BlockingRecv error: recv got EOF from 127.0.0.1:38678 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 07 19:42:46 dist-test-slave-vq2q krb5kdc[7299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131765, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
I20260807 19:42:46.850894 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:46.833937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38684 (local address 127.30.208.62:45093)
0807 19:42:46.834151 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:46.834158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:46.834377 (+ 219us) server_negotiation.cc:407] Connection header received
0807 19:42:46.834524 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:46.834528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:46.834627 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:46.834713 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:46.835943 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.836666 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.838038 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.838510 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.838617 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:46.845711 (+ 7094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:46.845717 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:46.845721 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:46.845764 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:46.848368 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:46.849104 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:46.849108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:46.849111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:46.849317 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:46.849673 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:46.849677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:46.849679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:46.849962 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:46.850199 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:46.850411 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 19:42:46.850600 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:42:46.855088 7349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:46.862178 7349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:46.885836 7914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:46.874991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38954 (local address 127.30.208.1:37733)
0807 19:42:46.876237 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:46.876246 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:46.878068 (+ 1822us) server_negotiation.cc:407] Connection header received
0807 19:42:46.878191 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:46.878201 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:46.878479 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:46.878677 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:46.879407 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.880786 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.883125 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.884400 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.884535 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:46.884770 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:46.885319 (+ 549us) server_negotiation.cc:299] Negotiation successful
0807 19:42:46.885504 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1084,"thread_start_us":913,"threads_started":1}
I20260807 19:42:46.885876 7912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:46.874459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:37733 (local address 127.0.0.1:38954)
0807 19:42:46.877208 (+ 2749us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:46.877227 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:46.877361 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:46.878711 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:46.878718 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:46.878750 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:46.879202 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:46.879221 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.880917 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.880922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:46.882960 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:46.882973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.884893 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.884900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:46.885049 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:46.885085 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:42:46.885274 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 19:42:46.885489 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2600,"thread_start_us":2424,"threads_started":1}
I20260807 19:42:46.886447 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:46.874207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:38301 (local address 127.0.0.1:57312)
0807 19:42:46.876400 (+ 2193us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:46.876448 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:46.876607 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:46.879521 (+ 2914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:46.879526 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:46.879555 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:46.879942 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:46.879956 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.881624 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.881629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:46.883688 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:46.883703 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.885602 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.885608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:46.885804 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:46.885838 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:42:46.885923 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 19:42:46.886055 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":1,"thread_start_us":606,"threads_started":1}
I20260807 19:42:46.887074 7913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:46.874464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57312 (local address 127.30.208.2:38301)
0807 19:42:46.878850 (+ 4386us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:46.878859 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:46.878887 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:42:46.878982 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:46.878989 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:46.879342 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:46.879517 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:46.880125 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.881480 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.883888 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.885363 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.886371 (+ 1008us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:46.886574 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:46.886653 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 19:42:46.886810 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4228,"thread_start_us":3413,"threads_started":1}
I20260807 19:42:46.887351 7916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:46.875172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:43607 (local address 127.0.0.1:50962)
0807 19:42:46.878440 (+ 3268us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:46.878453 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:46.878554 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:46.879128 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:46.879134 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:46.879165 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:46.879644 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:46.879661 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.882201 (+ 2540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.882206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:46.884384 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:46.884401 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.886585 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.886591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:46.886753 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:46.886784 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 19:42:46.886866 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 19:42:46.886994 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3158,"mutex_wait_us":20,"thread_start_us":3014,"threads_started":1}
I20260807 19:42:46.887365 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:46.875261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50962 (local address 127.30.208.3:43607)
0807 19:42:46.875775 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:46.875784 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:46.878563 (+ 2779us) server_negotiation.cc:407] Connection header received
0807 19:42:46.878669 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:46.878676 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:46.878894 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:46.879053 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:46.879836 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.882056 (+ 2220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.884567 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.886066 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.886172 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:46.886392 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:46.886917 (+ 525us) server_negotiation.cc:299] Negotiation successful
0807 19:42:46.887093 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":239,"threads_started":1}
I20260807 19:42:46.890329 7502 tablet_service.cc:1467] Processing CreateTablet for tablet 0d0e82c68fde4327ae957c6ef2f659ba (DEFAULT_TABLE table=test-table [id=ae0878946dbc44739b8238ec825238de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:46.890774 7812 tablet_service.cc:1467] Processing CreateTablet for tablet 0d0e82c68fde4327ae957c6ef2f659ba (DEFAULT_TABLE table=test-table [id=ae0878946dbc44739b8238ec825238de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:46.890774 7655 tablet_service.cc:1467] Processing CreateTablet for tablet 0d0e82c68fde4327ae957c6ef2f659ba (DEFAULT_TABLE table=test-table [id=ae0878946dbc44739b8238ec825238de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:46.891618 7502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d0e82c68fde4327ae957c6ef2f659ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:46.892131 7812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d0e82c68fde4327ae957c6ef2f659ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:46.892141 7655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d0e82c68fde4327ae957c6ef2f659ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:46.903349 7917 tablet_bootstrap.cc:492] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895: Bootstrap starting.
I20260807 19:42:46.906560 7919 tablet_bootstrap.cc:492] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52: Bootstrap starting.
I20260807 19:42:46.906910 7917 tablet_bootstrap.cc:654] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:46.907429 7918 tablet_bootstrap.cc:492] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554: Bootstrap starting.
I20260807 19:42:46.908878 7917 log.cc:824] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:46.911015 7918 tablet_bootstrap.cc:654] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:46.912103 7919 tablet_bootstrap.cc:654] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:46.912297 7918 log.cc:824] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:46.912357 7917 tablet_bootstrap.cc:492] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895: No bootstrap required, opened a new log
I20260807 19:42:46.912642 7917 ts_tablet_manager.cc:1397] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 19:42:46.913113 7919 log.cc:824] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:46.915638 7919 tablet_bootstrap.cc:492] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52: No bootstrap required, opened a new log
I20260807 19:42:46.915920 7919 ts_tablet_manager.cc:1397] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260807 19:42:46.918887 7918 tablet_bootstrap.cc:492] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554: No bootstrap required, opened a new log
I20260807 19:42:46.919203 7918 ts_tablet_manager.cc:1397] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.009s
I20260807 19:42:46.921348 7917 raft_consensus.cc:348] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:46.921718 7917 raft_consensus.cc:374] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:46.921834 7917 raft_consensus.cc:729] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92b8396827f24c6887da469b9b816895, State: Initialized, Role: FOLLOWER
I20260807 19:42:46.922449 7917 consensus_queue.cc:260] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:46.924218 7917 ts_tablet_manager.cc:1428] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895: Time spent starting tablet: real 0.011s user 0.013s sys 0.000s
I20260807 19:42:46.926568 7919 raft_consensus.cc:348] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:46.927099 7919 raft_consensus.cc:374] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:46.927238 7919 raft_consensus.cc:729] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2677256de76840bfbd379bcb63c4ea52, State: Initialized, Role: FOLLOWER
I20260807 19:42:46.927953 7919 consensus_queue.cc:260] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:46.929982 7919 ts_tablet_manager.cc:1428] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 19:42:46.929718 7918 raft_consensus.cc:348] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:46.930197 7918 raft_consensus.cc:374] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:46.930315 7918 raft_consensus.cc:729] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d6b390dce8b4ab5b978ea2286e63554, State: Initialized, Role: FOLLOWER
I20260807 19:42:46.930899 7918 consensus_queue.cc:260] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:46.932637 7918 ts_tablet_manager.cc:1428] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260807 19:42:46.938535 7733 tablet.cc:2369] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:46.950299 7925 raft_consensus.cc:482] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:46.950618 7925 raft_consensus.cc:504] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:46.952263 7925 leader_election.cc:288] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92b8396827f24c6887da469b9b816895 (127.30.208.1:37733), 2677256de76840bfbd379bcb63c4ea52 (127.30.208.3:43607)
I20260807 19:42:46.961462 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:46.952770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:39965 (local address 127.30.208.3:43607)
0807 19:42:46.952967 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:46.952972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:46.953728 (+ 756us) server_negotiation.cc:407] Connection header received
0807 19:42:46.953934 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:46.953939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:46.954031 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:46.954133 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:46.954986 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.956074 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.958958 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.960221 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.960379 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:46.960456 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:46.961032 (+ 576us) server_negotiation.cc:299] Negotiation successful
0807 19:42:46.961199 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:42:46.961462 7914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:46.952849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:45073 (local address 127.30.208.1:37733)
0807 19:42:46.953024 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:46.953030 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:46.953729 (+ 699us) server_negotiation.cc:407] Connection header received
0807 19:42:46.953943 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:46.953948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:46.954031 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:46.954132 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:46.954913 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.956044 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.958958 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.960191 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.960379 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:46.960456 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:46.961032 (+ 576us) server_negotiation.cc:299] Negotiation successful
0807 19:42:46.961207 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:42:46.961647 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:46.952557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:37733 (local address 127.30.208.2:45073)
0807 19:42:46.953597 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:46.953618 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:46.953815 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:46.954188 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:46.954193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:46.954243 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:46.954755 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:46.954773 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.956176 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.956181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:46.958756 (+ 2575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:46.958774 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.960501 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.960506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:46.960796 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:46.960856 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 19:42:46.960960 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:42:46.961140 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"mutex_wait_us":222,"spinlock_wait_cycles":46208,"thread_start_us":670,"threads_started":1}
I20260807 19:42:46.961957 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:46.952677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:43607 (local address 127.30.208.2:39965)
0807 19:42:46.953602 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:46.953618 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:46.953777 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:46.954188 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:46.954193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:46.954241 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:46.954784 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:46.954831 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.956221 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.956230 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:46.958755 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:46.958774 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:46.960486 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:46.960494 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:46.960790 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:46.960858 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 19:42:46.960960 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 19:42:46.961142 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"mutex_wait_us":195,"spinlock_wait_cycles":14336,"thread_start_us":529,"threads_started":1}
I20260807 19:42:46.962775 7524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d0e82c68fde4327ae957c6ef2f659ba" candidate_uuid: "9d6b390dce8b4ab5b978ea2286e63554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92b8396827f24c6887da469b9b816895" is_pre_election: true
I20260807 19:42:46.962776 7837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d0e82c68fde4327ae957c6ef2f659ba" candidate_uuid: "9d6b390dce8b4ab5b978ea2286e63554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2677256de76840bfbd379bcb63c4ea52" is_pre_election: true
I20260807 19:42:46.963253 7837 raft_consensus.cc:2454] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d6b390dce8b4ab5b978ea2286e63554 in term 0.
I20260807 19:42:46.963258 7524 raft_consensus.cc:2454] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d6b390dce8b4ab5b978ea2286e63554 in term 0.
I20260807 19:42:46.963886 7599 leader_election.cc:302] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2677256de76840bfbd379bcb63c4ea52, 92b8396827f24c6887da469b9b816895, 9d6b390dce8b4ab5b978ea2286e63554; no voters:
I20260807 19:42:46.964305 7925 raft_consensus.cc:2781] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:46.964460 7925 raft_consensus.cc:482] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:46.964567 7925 raft_consensus.cc:3037] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:46.966884 7925 raft_consensus.cc:504] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:46.967504 7925 leader_election.cc:288] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [CANDIDATE]: Term 1 election: Requested vote from peers 92b8396827f24c6887da469b9b816895 (127.30.208.1:37733), 2677256de76840bfbd379bcb63c4ea52 (127.30.208.3:43607)
I20260807 19:42:46.967998 7524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d0e82c68fde4327ae957c6ef2f659ba" candidate_uuid: "9d6b390dce8b4ab5b978ea2286e63554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92b8396827f24c6887da469b9b816895"
I20260807 19:42:46.968001 7837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d0e82c68fde4327ae957c6ef2f659ba" candidate_uuid: "9d6b390dce8b4ab5b978ea2286e63554" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2677256de76840bfbd379bcb63c4ea52"
I20260807 19:42:46.968186 7524 raft_consensus.cc:3037] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:46.968179 7837 raft_consensus.cc:3037] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:46.970428 7837 raft_consensus.cc:2454] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d6b390dce8b4ab5b978ea2286e63554 in term 1.
I20260807 19:42:46.970439 7524 raft_consensus.cc:2454] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d6b390dce8b4ab5b978ea2286e63554 in term 1.
I20260807 19:42:46.970845 7600 leader_election.cc:302] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2677256de76840bfbd379bcb63c4ea52, 9d6b390dce8b4ab5b978ea2286e63554; no voters:
I20260807 19:42:46.971154 7925 raft_consensus.cc:2781] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:46.971572 7925 raft_consensus.cc:686] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 LEADER]: Becoming Leader. State: Replica: 9d6b390dce8b4ab5b978ea2286e63554, State: Running, Role: LEADER
I20260807 19:42:46.972038 7925 consensus_queue.cc:237] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
W20260807 19:42:46.976857 7889 tablet.cc:2369] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:46.976701 7350 catalog_manager.cc:5495] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d6b390dce8b4ab5b978ea2286e63554 (127.30.208.2). New cstate: current_term: 1 leader_uuid: "9d6b390dce8b4ab5b978ea2286e63554" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } health_report { overall_health: UNKNOWN } } }
I20260807 19:42:47.014556 7913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.008434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57322 (local address 127.30.208.2:38301)
0807 19:42:47.008644 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:47.008649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:47.008729 (+ 80us) server_negotiation.cc:407] Connection header received
0807 19:42:47.008945 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:47.008949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:47.009039 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:47.009163 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:47.009566 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.010240 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.012158 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.012577 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.012672 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.013164 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:47.013945 (+ 781us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:47.014077 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:47.014198 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 19:42:47.014276 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:42:47.022758 7837 raft_consensus.cc:1261] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Refusing update from remote peer 9d6b390dce8b4ab5b978ea2286e63554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:47.022814 7524 raft_consensus.cc:1261] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Refusing update from remote peer 9d6b390dce8b4ab5b978ea2286e63554: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:47.023793 7928 consensus_queue.cc:1035] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:47.024149 7925 consensus_queue.cc:1035] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:47.054687 7349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:47.054961 7349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:47.058776 7349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f: Sending DeleteTablet for 3 replicas of tablet 0d0e82c68fde4327ae957c6ef2f659ba
I20260807 19:42:47.059923 7812 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d0e82c68fde4327ae957c6ef2f659ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:47 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:50962
I20260807 19:42:47.059929 7655 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d0e82c68fde4327ae957c6ef2f659ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:47 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:57312
I20260807 19:42:47.060722 7502 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d0e82c68fde4327ae957c6ef2f659ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:47 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:38954
I20260807 19:42:47.060873 7944 tablet_replica.cc:330] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52: stopping tablet replica
I20260807 19:42:47.061360 7944 raft_consensus.cc:2229] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:47.061890 7944 raft_consensus.cc:2258] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:47.062664 7946 tablet_replica.cc:330] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554: stopping tablet replica
I20260807 19:42:47.063112 7946 raft_consensus.cc:2229] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:47.063510 7349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 19:42:47.063750 7946 raft_consensus.cc:2258] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:47.064126 7944 ts_tablet_manager.cc:1905] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 19:42:47.064141 7349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:47.066004 7946 ts_tablet_manager.cc:1905] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:47.071326 7944 ts_tablet_manager.cc:1918] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:47.071558 7944 log.cc:1196] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/0d0e82c68fde4327ae957c6ef2f659ba
I20260807 19:42:47.072180 7944 ts_tablet_manager.cc:1939] T 0d0e82c68fde4327ae957c6ef2f659ba P 2677256de76840bfbd379bcb63c4ea52: Deleting consensus metadata
I20260807 19:42:47.073477 7946 ts_tablet_manager.cc:1918] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:47.073776 7946 log.cc:1196] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/0d0e82c68fde4327ae957c6ef2f659ba
I20260807 19:42:47.073820 7655 tablet_service.cc:1467] Processing CreateTablet for tablet 73b5f741dd09446e9c022a44e14ff958 (DEFAULT_TABLE table=test-table [id=95f8fc0eb9cf4a1786396b9607d48f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:47.074286 7655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b5f741dd09446e9c022a44e14ff958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:47.074456 7946 ts_tablet_manager.cc:1939] T 0d0e82c68fde4327ae957c6ef2f659ba P 9d6b390dce8b4ab5b978ea2286e63554: Deleting consensus metadata
I20260807 19:42:47.075217 7501 tablet_service.cc:1467] Processing CreateTablet for tablet 73b5f741dd09446e9c022a44e14ff958 (DEFAULT_TABLE table=test-table [id=95f8fc0eb9cf4a1786396b9607d48f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:47.075313 7331 catalog_manager.cc:4841] TS 2677256de76840bfbd379bcb63c4ea52 (127.30.208.3:43607): tablet 0d0e82c68fde4327ae957c6ef2f659ba (table test-table [id=ae0878946dbc44739b8238ec825238de]) successfully deleted
I20260807 19:42:47.075853 7501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b5f741dd09446e9c022a44e14ff958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:47.075928 7812 tablet_service.cc:1467] Processing CreateTablet for tablet 73b5f741dd09446e9c022a44e14ff958 (DEFAULT_TABLE table=test-table [id=95f8fc0eb9cf4a1786396b9607d48f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:47.076339 7812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73b5f741dd09446e9c022a44e14ff958. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:47.081749 7947 tablet_replica.cc:330] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895: stopping tablet replica
I20260807 19:42:47.082310 7947 raft_consensus.cc:2229] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:47.082886 7947 raft_consensus.cc:2258] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:47.083014 7919 tablet_bootstrap.cc:492] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52: Bootstrap starting.
I20260807 19:42:47.084625 7919 tablet_bootstrap.cc:654] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:47.085297 7947 ts_tablet_manager.cc:1905] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:47.085410 7917 tablet_bootstrap.cc:492] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895: Bootstrap starting.
I20260807 19:42:47.087428 7917 tablet_bootstrap.cc:654] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:47.092310 7947 ts_tablet_manager.cc:1918] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:47.092545 7947 log.cc:1196] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/0d0e82c68fde4327ae957c6ef2f659ba
I20260807 19:42:47.093214 7947 ts_tablet_manager.cc:1939] T 0d0e82c68fde4327ae957c6ef2f659ba P 92b8396827f24c6887da469b9b816895: Deleting consensus metadata
I20260807 19:42:47.095036 7917 tablet_bootstrap.cc:492] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895: No bootstrap required, opened a new log
I20260807 19:42:47.095108 7330 catalog_manager.cc:4841] TS 92b8396827f24c6887da469b9b816895 (127.30.208.1:37733): tablet 0d0e82c68fde4327ae957c6ef2f659ba (table test-table [id=ae0878946dbc44739b8238ec825238de]) successfully deleted
I20260807 19:42:47.095252 7917 ts_tablet_manager.cc:1397] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.004s
I20260807 19:42:47.096063 7917 raft_consensus.cc:348] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.096376 7917 raft_consensus.cc:374] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:47.096519 7917 raft_consensus.cc:729] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92b8396827f24c6887da469b9b816895, State: Initialized, Role: FOLLOWER
I20260807 19:42:47.096848 7917 consensus_queue.cc:260] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.097163 7333 catalog_manager.cc:4841] TS 9d6b390dce8b4ab5b978ea2286e63554 (127.30.208.2:38301): tablet 0d0e82c68fde4327ae957c6ef2f659ba (table test-table [id=ae0878946dbc44739b8238ec825238de]) successfully deleted
I20260807 19:42:47.098637 7917 ts_tablet_manager.cc:1428] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 19:42:47.100576 7919 tablet_bootstrap.cc:492] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52: No bootstrap required, opened a new log
I20260807 19:42:47.100785 7919 ts_tablet_manager.cc:1397] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.011s
I20260807 19:42:47.101217 7918 tablet_bootstrap.cc:492] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554: Bootstrap starting.
I20260807 19:42:47.101701 7919 raft_consensus.cc:348] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.101938 7919 raft_consensus.cc:374] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:47.102025 7919 raft_consensus.cc:729] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2677256de76840bfbd379bcb63c4ea52, State: Initialized, Role: FOLLOWER
I20260807 19:42:47.102295 7919 consensus_queue.cc:260] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.102808 7918 tablet_bootstrap.cc:654] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:47.102910 7919 ts_tablet_manager.cc:1428] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 19:42:47.106379 7918 tablet_bootstrap.cc:492] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554: No bootstrap required, opened a new log
I20260807 19:42:47.106549 7918 ts_tablet_manager.cc:1397] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 19:42:47.107496 7918 raft_consensus.cc:348] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.107688 7918 raft_consensus.cc:374] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:47.107805 7918 raft_consensus.cc:729] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d6b390dce8b4ab5b978ea2286e63554, State: Initialized, Role: FOLLOWER
I20260807 19:42:47.108144 7918 consensus_queue.cc:260] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.108939 7918 ts_tablet_manager.cc:1428] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
W20260807 19:42:47.154713 7577 tablet.cc:2369] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:42:47.233549 7923 raft_consensus.cc:482] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:47.233788 7923 raft_consensus.cc:504] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.235383 7923 leader_election.cc:288] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d6b390dce8b4ab5b978ea2286e63554 (127.30.208.2:38301), 2677256de76840bfbd379bcb63c4ea52 (127.30.208.3:43607)
I20260807 19:42:47.243106 7913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.236380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:50729 (local address 127.30.208.2:38301)
0807 19:42:47.236569 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:47.236575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:47.236596 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:42:47.236815 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:47.236823 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:47.236908 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:47.236967 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:47.237761 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.238982 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.240902 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.241934 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.242084 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.242163 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:47.242633 (+ 470us) server_negotiation.cc:299] Negotiation successful
0807 19:42:47.242780 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:42:47.243153 7952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.235733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:38301 (local address 127.30.208.1:50729)
0807 19:42:47.236415 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:47.236439 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:47.236603 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:47.237222 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:47.237227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:47.237253 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:47.237598 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.237612 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.239127 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.239132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.240744 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.240757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.242257 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.242264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.242451 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.242487 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:42:47.242584 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 19:42:47.242720 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":72,"spinlock_wait_cycles":35968,"thread_start_us":862,"threads_started":1}
I20260807 19:42:47.243861 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.235906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:49023 (local address 127.30.208.3:43607)
0807 19:42:47.236171 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:47.236178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:47.237333 (+ 1155us) server_negotiation.cc:407] Connection header received
0807 19:42:47.237544 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:47.237550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:47.237640 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:47.237734 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:47.238305 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.239011 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.241566 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.242605 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.242759 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.242823 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:47.243455 (+ 632us) server_negotiation.cc:299] Negotiation successful
0807 19:42:47.243604 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260807 19:42:47.244606 7953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.235873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:43607 (local address 127.30.208.1:49023)
0807 19:42:47.237214 (+ 1341us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:47.237232 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:47.237404 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:47.237791 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:47.237795 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:47.237818 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:47.238139 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.238148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.239169 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.239174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.241406 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.241422 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.242977 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.242983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.243240 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.243319 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 19:42:47.244119 (+ 800us) client_negotiation.cc:240] Negotiation successful
0807 19:42:47.244226 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1163,"spinlock_wait_cycles":768,"thread_start_us":1014,"threads_started":1}
I20260807 19:42:47.244632 7682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b5f741dd09446e9c022a44e14ff958" candidate_uuid: "92b8396827f24c6887da469b9b816895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d6b390dce8b4ab5b978ea2286e63554" is_pre_election: true
I20260807 19:42:47.245150 7682 raft_consensus.cc:2454] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92b8396827f24c6887da469b9b816895 in term 0.
I20260807 19:42:47.245354 7837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b5f741dd09446e9c022a44e14ff958" candidate_uuid: "92b8396827f24c6887da469b9b816895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2677256de76840bfbd379bcb63c4ea52" is_pre_election: true
I20260807 19:42:47.245587 7837 raft_consensus.cc:2454] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92b8396827f24c6887da469b9b816895 in term 0.
I20260807 19:42:47.245827 7449 leader_election.cc:302] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92b8396827f24c6887da469b9b816895, 9d6b390dce8b4ab5b978ea2286e63554; no voters:
I20260807 19:42:47.246249 7923 raft_consensus.cc:2781] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:47.246374 7923 raft_consensus.cc:482] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:47.246469 7923 raft_consensus.cc:3037] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:47.248610 7923 raft_consensus.cc:504] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.249291 7923 leader_election.cc:288] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [CANDIDATE]: Term 1 election: Requested vote from peers 9d6b390dce8b4ab5b978ea2286e63554 (127.30.208.2:38301), 2677256de76840bfbd379bcb63c4ea52 (127.30.208.3:43607)
I20260807 19:42:47.249939 7837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b5f741dd09446e9c022a44e14ff958" candidate_uuid: "92b8396827f24c6887da469b9b816895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2677256de76840bfbd379bcb63c4ea52"
I20260807 19:42:47.250020 7682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73b5f741dd09446e9c022a44e14ff958" candidate_uuid: "92b8396827f24c6887da469b9b816895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d6b390dce8b4ab5b978ea2286e63554"
I20260807 19:42:47.250149 7837 raft_consensus.cc:3037] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:47.250223 7682 raft_consensus.cc:3037] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:47.252604 7682 raft_consensus.cc:2454] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92b8396827f24c6887da469b9b816895 in term 1.
I20260807 19:42:47.252986 7449 leader_election.cc:302] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92b8396827f24c6887da469b9b816895, 9d6b390dce8b4ab5b978ea2286e63554; no voters:
I20260807 19:42:47.253123 7837 raft_consensus.cc:2454] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92b8396827f24c6887da469b9b816895 in term 1.
I20260807 19:42:47.253321 7923 raft_consensus.cc:2781] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:47.253830 7923 raft_consensus.cc:686] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [term 1 LEADER]: Becoming Leader. State: Replica: 92b8396827f24c6887da469b9b816895, State: Running, Role: LEADER
I20260807 19:42:47.254282 7923 consensus_queue.cc:237] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.257977 7349 catalog_manager.cc:5495] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92b8396827f24c6887da469b9b816895 (127.30.208.1). New cstate: current_term: 1 leader_uuid: "92b8396827f24c6887da469b9b816895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } health_report { overall_health: UNKNOWN } } }
Aug 07 19:42:47 dist-test-slave-vq2q krb5kdc[7299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131763, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.62@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
I20260807 19:42:47.327086 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.311714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:45093 (local address 127.0.0.1:38686)
0807 19:42:47.312366 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:47.312398 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:47.312556 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:47.312997 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:47.313001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:47.313986 (+ 985us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:47.314353 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.314367 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.315424 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.315430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.316885 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.316902 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.317686 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.317694 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.317869 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.318572 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:47.318603 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:47.321335 (+ 2732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:47.324343 (+ 3008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:47.324359 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:47.324367 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:47.324785 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:47.325123 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:47.325128 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:47.325132 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:47.325325 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:47.325893 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:47.325933 (+ 40us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:47.326154 (+ 221us) client_negotiation.cc:769] Sending connection context
0807 19:42:47.326329 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 19:42:47.326624 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":340,"threads_started":1}
I20260807 19:42:47.327108 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.311742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38686 (local address 127.30.208.62:45093)
0807 19:42:47.312000 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:47.312007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:47.312509 (+ 502us) server_negotiation.cc:407] Connection header received
0807 19:42:47.312757 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:47.312761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:47.312828 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:47.312921 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:47.314526 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.315295 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.317056 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.317490 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.317581 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.321528 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:47.321533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:47.321536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:47.321568 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:47.324177 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:47.324897 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:47.324902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:47.324906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:47.325024 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:47.325423 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:47.325428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:47.325431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:47.325740 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:47.325926 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:47.326555 (+ 629us) server_negotiation.cc:299] Negotiation successful
0807 19:42:47.326782 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260807 19:42:47.332469 7349 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:38686:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 19:42:47.333549 7349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:47.340739 7501 tablet_service.cc:1467] Processing CreateTablet for tablet 02e661d17eca4e319fc0468299a52fe6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd337471bee744e2949b61f0ce55ad4b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:42:47.341341 7501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02e661d17eca4e319fc0468299a52fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:47.341388 7655 tablet_service.cc:1467] Processing CreateTablet for tablet 02e661d17eca4e319fc0468299a52fe6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd337471bee744e2949b61f0ce55ad4b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:42:47.342008 7655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02e661d17eca4e319fc0468299a52fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:47.346894 7917 tablet_bootstrap.cc:492] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895: Bootstrap starting.
I20260807 19:42:47.348353 7917 tablet_bootstrap.cc:654] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:47.349779 7812 tablet_service.cc:1467] Processing CreateTablet for tablet 02e661d17eca4e319fc0468299a52fe6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fd337471bee744e2949b61f0ce55ad4b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 19:42:47.350390 7812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02e661d17eca4e319fc0468299a52fe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:47.350409 7917 tablet_bootstrap.cc:492] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895: No bootstrap required, opened a new log
I20260807 19:42:47.350551 7917 ts_tablet_manager.cc:1397] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 19:42:47.351126 7917 raft_consensus.cc:348] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.351320 7917 raft_consensus.cc:374] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:47.351394 7917 raft_consensus.cc:729] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92b8396827f24c6887da469b9b816895, State: Initialized, Role: FOLLOWER
I20260807 19:42:47.351624 7917 consensus_queue.cc:260] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.352034 7923 tablet_replica.cc:439] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } } }
I20260807 19:42:47.352221 7917 ts_tablet_manager.cc:1428] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 19:42:47.352358 7923 tablet_replica.cc:442] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:42:47.354454 7918 tablet_bootstrap.cc:492] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554: Bootstrap starting.
I20260807 19:42:47.356053 7918 tablet_bootstrap.cc:654] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:47.358187 7919 tablet_bootstrap.cc:492] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52: Bootstrap starting.
I20260807 19:42:47.359797 7919 tablet_bootstrap.cc:654] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:47.359962 7918 tablet_bootstrap.cc:492] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554: No bootstrap required, opened a new log
I20260807 19:42:47.360136 7918 ts_tablet_manager.cc:1397] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20260807 19:42:47.360982 7918 raft_consensus.cc:348] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.361289 7918 raft_consensus.cc:374] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:47.361385 7918 raft_consensus.cc:729] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d6b390dce8b4ab5b978ea2286e63554, State: Initialized, Role: FOLLOWER
I20260807 19:42:47.361660 7918 consensus_queue.cc:260] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.362361 7918 ts_tablet_manager.cc:1428] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 19:42:47.362170 7928 tablet_replica.cc:439] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } } }
I20260807 19:42:47.362571 7928 tablet_replica.cc:442] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:42:47.364012 7919 tablet_bootstrap.cc:492] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52: No bootstrap required, opened a new log
I20260807 19:42:47.364184 7919 ts_tablet_manager.cc:1397] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 19:42:47.364949 7919 raft_consensus.cc:348] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.365213 7919 raft_consensus.cc:374] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:47.365303 7919 raft_consensus.cc:729] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2677256de76840bfbd379bcb63c4ea52, State: Initialized, Role: FOLLOWER
I20260807 19:42:47.365554 7919 consensus_queue.cc:260] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.366168 7919 ts_tablet_manager.cc:1428] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 19:42:47.366021 7924 tablet_replica.cc:439] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } } }
I20260807 19:42:47.366410 7924 tablet_replica.cc:442] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:42:47.571394 7924 raft_consensus.cc:482] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:47.571638 7924 raft_consensus.cc:504] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.573203 7924 leader_election.cc:288] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92b8396827f24c6887da469b9b816895 (127.30.208.1:37733), 9d6b390dce8b4ab5b978ea2286e63554 (127.30.208.2:38301)
I20260807 19:42:47.579595 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.573680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:49041 (local address 127.30.208.1:37733)
0807 19:42:47.574408 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:47.574413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:47.574430 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:42:47.574489 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:47.574492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:47.574556 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:47.574640 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:47.575262 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.576000 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.577729 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.578559 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.578716 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.578788 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:47.579140 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 19:42:47.579294 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"spinlock_wait_cycles":588160,"thread_start_us":337,"threads_started":1}
I20260807 19:42:47.579620 7913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.573651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:42281 (local address 127.30.208.2:38301)
0807 19:42:47.573832 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:47.573838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:47.574481 (+ 643us) server_negotiation.cc:407] Connection header received
0807 19:42:47.574640 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:47.574644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:47.574705 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:47.574780 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:42:47.575411 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.576084 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.577798 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.578659 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.578819 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.578890 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:47.579208 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 19:42:47.579364 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:42:47.579802 7968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.573529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:38301 (local address 127.30.208.3:42281)
0807 19:42:47.574359 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:47.574378 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:47.574509 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:47.574824 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:47.574829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:47.574870 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:47.575249 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.575259 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.576215 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.576220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.577649 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.577662 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.578875 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.578879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.579006 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.579028 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 19:42:47.579158 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 19:42:47.579294 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":330,"thread_start_us":260,"threads_started":1}
I20260807 19:42:47.580111 7966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.573521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:37733 (local address 127.30.208.3:49041)
0807 19:42:47.574065 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:47.574085 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:47.574241 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:47.574676 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:47.574680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:47.574716 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:42:47.575081 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.575091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.576136 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.576140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.577580 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.577593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.578760 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.578766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.578961 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.578985 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:42:47.579094 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 19:42:47.579275 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":5,"spinlock_wait_cycles":8448,"thread_start_us":299,"threads_started":1}
I20260807 19:42:47.580404 7682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02e661d17eca4e319fc0468299a52fe6" candidate_uuid: "2677256de76840bfbd379bcb63c4ea52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d6b390dce8b4ab5b978ea2286e63554" is_pre_election: true
I20260807 19:42:47.580664 7682 raft_consensus.cc:2454] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2677256de76840bfbd379bcb63c4ea52 in term 0.
I20260807 19:42:47.580729 7524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02e661d17eca4e319fc0468299a52fe6" candidate_uuid: "2677256de76840bfbd379bcb63c4ea52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92b8396827f24c6887da469b9b816895" is_pre_election: true
I20260807 19:42:47.580948 7524 raft_consensus.cc:2454] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2677256de76840bfbd379bcb63c4ea52 in term 0.
I20260807 19:42:47.581208 7758 leader_election.cc:302] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2677256de76840bfbd379bcb63c4ea52, 9d6b390dce8b4ab5b978ea2286e63554; no voters:
I20260807 19:42:47.581634 7924 raft_consensus.cc:2781] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:47.581756 7924 raft_consensus.cc:482] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:47.581854 7924 raft_consensus.cc:3037] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:47.583963 7924 raft_consensus.cc:504] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.584558 7924 leader_election.cc:288] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [CANDIDATE]: Term 1 election: Requested vote from peers 92b8396827f24c6887da469b9b816895 (127.30.208.1:37733), 9d6b390dce8b4ab5b978ea2286e63554 (127.30.208.2:38301)
I20260807 19:42:47.584985 7524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02e661d17eca4e319fc0468299a52fe6" candidate_uuid: "2677256de76840bfbd379bcb63c4ea52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92b8396827f24c6887da469b9b816895"
I20260807 19:42:47.585033 7682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02e661d17eca4e319fc0468299a52fe6" candidate_uuid: "2677256de76840bfbd379bcb63c4ea52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d6b390dce8b4ab5b978ea2286e63554"
I20260807 19:42:47.585170 7524 raft_consensus.cc:3037] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:47.585255 7682 raft_consensus.cc:3037] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:47.587134 7682 raft_consensus.cc:2454] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2677256de76840bfbd379bcb63c4ea52 in term 1.
I20260807 19:42:47.587208 7524 raft_consensus.cc:2454] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2677256de76840bfbd379bcb63c4ea52 in term 1.
I20260807 19:42:47.587503 7758 leader_election.cc:302] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2677256de76840bfbd379bcb63c4ea52, 9d6b390dce8b4ab5b978ea2286e63554; no voters:
I20260807 19:42:47.587864 7924 raft_consensus.cc:2781] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:47.588199 7924 raft_consensus.cc:686] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [term 1 LEADER]: Becoming Leader. State: Replica: 2677256de76840bfbd379bcb63c4ea52, State: Running, Role: LEADER
I20260807 19:42:47.588588 7924 consensus_queue.cc:237] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.590657 7969 tablet_replica.cc:439] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52: TxnStatusTablet state changed. Reason: New leader 2677256de76840bfbd379bcb63c4ea52. Latest consensus state: current_term: 1 leader_uuid: "2677256de76840bfbd379bcb63c4ea52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } } }
I20260807 19:42:47.591059 7969 tablet_replica.cc:442] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:42:47.591715 7971 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 19:42:47.591928 7971 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 19:42:47.592135 7349 catalog_manager.cc:5495] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2677256de76840bfbd379bcb63c4ea52 (127.30.208.3). New cstate: current_term: 1 leader_uuid: "2677256de76840bfbd379bcb63c4ea52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } health_report { overall_health: HEALTHY } } }
I20260807 19:42:47.610159 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.603878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50976 (local address 127.30.208.3:43607)
0807 19:42:47.604387 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:47.604391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:47.604406 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:42:47.604471 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:47.604475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:47.604543 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:47.604643 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:47.605092 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.605952 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.607902 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.608413 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.608535 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.609054 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:47.609622 (+ 568us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:47.609728 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:47.609870 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 19:42:47.609941 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393}
I20260807 19:42:47.610307 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.603823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:43607 (local address 127.0.0.1:50976)
0807 19:42:47.604037 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:47.604054 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:47.604161 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:47.604729 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:47.604734 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:47.604749 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:42:47.604954 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.604966 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.606085 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.606091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.607738 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.607755 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.608591 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.608597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.608803 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.608881 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:42:47.609736 (+ 855us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:42:47.609760 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:42:47.609855 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 19:42:47.610016 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 19:42:47.614912 7524 raft_consensus.cc:1261] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Refusing update from remote peer 2677256de76840bfbd379bcb63c4ea52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:47.615190 7682 raft_consensus.cc:1261] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Refusing update from remote peer 2677256de76840bfbd379bcb63c4ea52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:47.615844 7969 consensus_queue.cc:1035] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:47.616137 7924 consensus_queue.cc:1035] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:47.619653 7928 tablet_replica.cc:439] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554: TxnStatusTablet state changed. Reason: New leader 2677256de76840bfbd379bcb63c4ea52. Latest consensus state: current_term: 1 leader_uuid: "2677256de76840bfbd379bcb63c4ea52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } } }
I20260807 19:42:47.619962 7928 tablet_replica.cc:442] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:42:47.623201 7924 tablet_replica.cc:439] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2677256de76840bfbd379bcb63c4ea52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } } }
I20260807 19:42:47.623510 7924 tablet_replica.cc:442] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:42:47.633584 7928 tablet_replica.cc:439] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2677256de76840bfbd379bcb63c4ea52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } } }
I20260807 19:42:47.633885 7928 tablet_replica.cc:442] T 02e661d17eca4e319fc0468299a52fe6 P 9d6b390dce8b4ab5b978ea2286e63554: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:42:47.641160 7923 tablet_replica.cc:439] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895: TxnStatusTablet state changed. Reason: New leader 2677256de76840bfbd379bcb63c4ea52. Latest consensus state: current_term: 1 leader_uuid: "2677256de76840bfbd379bcb63c4ea52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } } }
I20260807 19:42:47.641542 7923 tablet_replica.cc:442] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:42:47.643257 7969 tablet_replica.cc:439] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2677256de76840bfbd379bcb63c4ea52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } } }
I20260807 19:42:47.643601 7969 tablet_replica.cc:442] T 02e661d17eca4e319fc0468299a52fe6 P 2677256de76840bfbd379bcb63c4ea52: This TxnStatusTablet replica's current role is: LEADER
I20260807 19:42:47.646438 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.634001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38956 (local address 127.30.208.1:37733)
0807 19:42:47.634521 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:47.634527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:47.637700 (+ 3173us) server_negotiation.cc:407] Connection header received
0807 19:42:47.637941 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:47.637945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:47.638048 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:47.638184 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:47.638617 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.639327 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.640743 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.641408 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.641698 (+ 290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.642155 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:47.645458 (+ 3303us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:47.645596 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:47.646052 (+ 456us) server_negotiation.cc:299] Negotiation successful
0807 19:42:47.646140 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376}
I20260807 19:42:47.647222 7923 tablet_replica.cc:439] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2677256de76840bfbd379bcb63c4ea52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } } }
I20260807 19:42:47.647537 7923 tablet_replica.cc:442] T 02e661d17eca4e319fc0468299a52fe6 P 92b8396827f24c6887da469b9b816895: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 19:42:47.663882 7978 mvcc.cc:205] Tried to move back new op lower bound from 7315995720143237120 to 7315995720046424064. Current Snapshot: MvccSnapshot[applied={T|T < 7315995720143237120 or (T in {7315995720143237120})}]
I20260807 19:42:47.671278 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.663391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:40779 (local address 127.30.208.3:43607)
0807 19:42:47.663594 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:47.663604 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:47.664036 (+ 432us) server_negotiation.cc:407] Connection header received
0807 19:42:47.664860 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:47.664866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:47.664959 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:47.665053 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:47.665688 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.666851 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.669060 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.669816 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.669933 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.670479 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:47.670628 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:47.670712 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:47.670883 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 19:42:47.670967 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:42:47.671424 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.662938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:43607 (local address 127.30.208.1:40779)
0807 19:42:47.663923 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:47.663950 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:47.664637 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:47.665091 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:47.665097 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:47.665111 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:42:47.665507 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.665520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.667050 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.667056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.668895 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.668918 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.670010 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.670017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.670209 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.670286 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:42:47.670736 (+ 450us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:42:47.670755 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 19:42:47.670844 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 19:42:47.670995 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":625,"threads_started":1}
I20260807 19:42:47.682161 7837 raft_consensus.cc:1261] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Refusing update from remote peer 92b8396827f24c6887da469b9b816895: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:47.682181 7682 raft_consensus.cc:1261] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Refusing update from remote peer 92b8396827f24c6887da469b9b816895: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:47.682857 7954 consensus_queue.cc:1035] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:47.683225 7923 consensus_queue.cc:1035] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:47.699007 7979 mvcc.cc:205] Tried to move back new op lower bound from 7315995720419930112 to 7315995718677495808. Current Snapshot: MvccSnapshot[applied={T|T < 7315995720419930112}]
I20260807 19:42:47.723922 7967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.716782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:48469 (local address 127.30.208.1:37733)
0807 19:42:47.716940 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:47.716946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:47.717661 (+ 715us) server_negotiation.cc:407] Connection header received
0807 19:42:47.717846 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:47.717852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:47.717951 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:47.718054 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:47.718901 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.720094 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.721725 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.722409 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.722533 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.723080 (+ 547us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:47.723232 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:47.723308 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:47.723545 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 19:42:47.723634 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 19:42:47.724743 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.716749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:37733 (local address 127.30.208.3:48469)
0807 19:42:47.717550 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:47.717571 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:47.717718 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:47.718083 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:47.718088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:47.718101 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 19:42:47.718519 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.718529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.720225 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.720229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.721580 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:47.721594 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.722614 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.722621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:47.722787 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.722871 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 19:42:47.723393 (+ 522us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 19:42:47.723411 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 19:42:47.724178 (+ 767us) client_negotiation.cc:240] Negotiation successful
0807 19:42:47.724342 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":283,"threads_started":1}
I20260807 19:42:47.771124 7350 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:47.771413 7350 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:47.778950 7350 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f: Sending DeleteTablet for 3 replicas of tablet 73b5f741dd09446e9c022a44e14ff958
I20260807 19:42:47.780577 7501 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b5f741dd09446e9c022a44e14ff958 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:47 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:38954
I20260807 19:42:47.782938 7655 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b5f741dd09446e9c022a44e14ff958 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:47 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:57312
I20260807 19:42:47.782142 8001 tablet_replica.cc:330] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895: stopping tablet replica
I20260807 19:42:47.783442 8001 raft_consensus.cc:2229] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:47.784039 8001 raft_consensus.cc:2258] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:47.785055 7812 tablet_service.cc:1514] Processing DeleteTablet for tablet 73b5f741dd09446e9c022a44e14ff958 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:47 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:50962
I20260807 19:42:47.792380 8001 ts_tablet_manager.cc:1905] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:47.801343 8002 tablet_replica.cc:330] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554: stopping tablet replica
I20260807 19:42:47.804927 8001 ts_tablet_manager.cc:1918] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:42:47.805344 8002 raft_consensus.cc:2229] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:47.805718 8002 raft_consensus.cc:2258] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:47.806115 8001 log.cc:1196] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/73b5f741dd09446e9c022a44e14ff958
I20260807 19:42:47.807040 8002 ts_tablet_manager.cc:1905] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:47.814451 8002 ts_tablet_manager.cc:1918] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:42:47.815160 8001 ts_tablet_manager.cc:1939] T 73b5f741dd09446e9c022a44e14ff958 P 92b8396827f24c6887da469b9b816895: Deleting consensus metadata
I20260807 19:42:47.816622 7330 catalog_manager.cc:4841] TS 92b8396827f24c6887da469b9b816895 (127.30.208.1:37733): tablet 73b5f741dd09446e9c022a44e14ff958 (table test-table [id=95f8fc0eb9cf4a1786396b9607d48f85]) successfully deleted
I20260807 19:42:47.817410 8003 tablet_replica.cc:330] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52: stopping tablet replica
I20260807 19:42:47.817708 8003 raft_consensus.cc:2229] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:47.818192 8003 raft_consensus.cc:2258] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:47.818470 8002 log.cc:1196] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/73b5f741dd09446e9c022a44e14ff958
I20260807 19:42:47.819237 8003 ts_tablet_manager.cc:1905] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:47.825809 8002 ts_tablet_manager.cc:1939] T 73b5f741dd09446e9c022a44e14ff958 P 9d6b390dce8b4ab5b978ea2286e63554: Deleting consensus metadata
I20260807 19:42:47.826516 8003 ts_tablet_manager.cc:1918] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:42:47.826650 8003 log.cc:1196] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/73b5f741dd09446e9c022a44e14ff958
I20260807 19:42:47.827059 8003 ts_tablet_manager.cc:1939] T 73b5f741dd09446e9c022a44e14ff958 P 2677256de76840bfbd379bcb63c4ea52: Deleting consensus metadata
I20260807 19:42:47.827334 7333 catalog_manager.cc:4841] TS 9d6b390dce8b4ab5b978ea2286e63554 (127.30.208.2:38301): tablet 73b5f741dd09446e9c022a44e14ff958 (table test-table [id=95f8fc0eb9cf4a1786396b9607d48f85]) successfully deleted
I20260807 19:42:47.828241 7331 catalog_manager.cc:4841] TS 2677256de76840bfbd379bcb63c4ea52 (127.30.208.3:43607): tablet 73b5f741dd09446e9c022a44e14ff958 (table test-table [id=95f8fc0eb9cf4a1786396b9607d48f85]) successfully deleted
Aug 07 19:42:47 dist-test-slave-vq2q krb5kdc[7299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 19:42:47 dist-test-slave-vq2q krb5kdc[7299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:47 dist-test-slave-vq2q krb5kdc[7299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:42:47.869958 8011 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:47.858534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38688 (local address 127.30.208.62:45093)
0807 19:42:47.859321 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:47.859327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:47.859347 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:42:47.859413 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:47.859418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:47.859498 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:47.859598 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:47.860912 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.861691 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.862976 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.863421 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.864055 (+ 634us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.869676 (+ 5621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38688: BlockingRecv error: recv got EOF from 127.0.0.1:38688 (error 108)
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":562,"threads_started":1}
Aug 07 19:42:47 dist-test-slave-vq2q krb5kdc[7299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM
I20260807 19:42:47.885582 8011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:47.872495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38692 (local address 127.30.208.62:45093)
0807 19:42:47.872648 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:47.872653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:47.872672 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:42:47.872853 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:47.872858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:47.872941 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:47.872998 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:47.874311 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.874986 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.876180 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:47.876580 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:47.876688 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:47.880482 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:47.880487 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:47.880490 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:47.880523 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:47.883023 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:47.883644 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:47.883650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:47.883652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:47.883804 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:47.884168 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:47.884173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:47.884176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:47.884437 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:47.884668 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:47.884979 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 19:42:47.885224 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:42:47.889029 7350 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:47.889678 7350 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:47.896559 7501 tablet_service.cc:1467] Processing CreateTablet for tablet 2c6b3c17a0814ae0b155b6fdfcc9799d (DEFAULT_TABLE table=test-table [id=08db99312a814d2a86f606b49d6be646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:47.897104 7501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c6b3c17a0814ae0b155b6fdfcc9799d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:47.897049 7812 tablet_service.cc:1467] Processing CreateTablet for tablet 2c6b3c17a0814ae0b155b6fdfcc9799d (DEFAULT_TABLE table=test-table [id=08db99312a814d2a86f606b49d6be646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:47.897718 7812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c6b3c17a0814ae0b155b6fdfcc9799d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:47.900204 7655 tablet_service.cc:1467] Processing CreateTablet for tablet 2c6b3c17a0814ae0b155b6fdfcc9799d (DEFAULT_TABLE table=test-table [id=08db99312a814d2a86f606b49d6be646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:47.900612 7655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c6b3c17a0814ae0b155b6fdfcc9799d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:47.909792 8017 tablet_bootstrap.cc:492] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895: Bootstrap starting.
I20260807 19:42:47.912015 8017 tablet_bootstrap.cc:654] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:47.914713 8018 tablet_bootstrap.cc:492] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52: Bootstrap starting.
I20260807 19:42:47.916703 8018 tablet_bootstrap.cc:654] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:47.918838 8019 tablet_bootstrap.cc:492] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554: Bootstrap starting.
I20260807 19:42:47.920547 8019 tablet_bootstrap.cc:654] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:47.923005 8019 tablet_bootstrap.cc:492] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554: No bootstrap required, opened a new log
I20260807 19:42:47.923173 8019 ts_tablet_manager.cc:1397] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 19:42:47.923684 8017 tablet_bootstrap.cc:492] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895: No bootstrap required, opened a new log
I20260807 19:42:47.923930 8017 ts_tablet_manager.cc:1397] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.005s
I20260807 19:42:47.923939 8019 raft_consensus.cc:348] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.924211 8019 raft_consensus.cc:374] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:47.924312 8019 raft_consensus.cc:729] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d6b390dce8b4ab5b978ea2286e63554, State: Initialized, Role: FOLLOWER
I20260807 19:42:47.925226 8018 tablet_bootstrap.cc:492] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52: No bootstrap required, opened a new log
I20260807 19:42:47.925396 8018 ts_tablet_manager.cc:1397] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.001s
I20260807 19:42:47.926239 8018 raft_consensus.cc:348] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.926507 8018 raft_consensus.cc:374] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:47.925014 8017 raft_consensus.cc:348] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.926604 8018 raft_consensus.cc:729] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2677256de76840bfbd379bcb63c4ea52, State: Initialized, Role: FOLLOWER
I20260807 19:42:47.926749 8017 raft_consensus.cc:374] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:47.926911 8017 raft_consensus.cc:729] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92b8396827f24c6887da469b9b816895, State: Initialized, Role: FOLLOWER
I20260807 19:42:47.926903 8019 consensus_queue.cc:260] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.927328 8017 consensus_queue.cc:260] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.928076 8017 ts_tablet_manager.cc:1428] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 19:42:47.930991 8019 ts_tablet_manager.cc:1428] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554: Time spent starting tablet: real 0.008s user 0.003s sys 0.000s
I20260807 19:42:47.931311 8018 consensus_queue.cc:260] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.935719 8018 ts_tablet_manager.cc:1428] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260807 19:42:47.978780 7969 raft_consensus.cc:482] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:47.979017 7969 raft_consensus.cc:504] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.979858 7969 leader_election.cc:288] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92b8396827f24c6887da469b9b816895 (127.30.208.1:37733), 9d6b390dce8b4ab5b978ea2286e63554 (127.30.208.2:38301)
I20260807 19:42:47.980633 7682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6b3c17a0814ae0b155b6fdfcc9799d" candidate_uuid: "2677256de76840bfbd379bcb63c4ea52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d6b390dce8b4ab5b978ea2286e63554" is_pre_election: true
I20260807 19:42:47.980647 7524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6b3c17a0814ae0b155b6fdfcc9799d" candidate_uuid: "2677256de76840bfbd379bcb63c4ea52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92b8396827f24c6887da469b9b816895" is_pre_election: true
I20260807 19:42:47.980924 7682 raft_consensus.cc:2454] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2677256de76840bfbd379bcb63c4ea52 in term 0.
I20260807 19:42:47.980931 7524 raft_consensus.cc:2454] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2677256de76840bfbd379bcb63c4ea52 in term 0.
I20260807 19:42:47.981525 7758 leader_election.cc:302] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2677256de76840bfbd379bcb63c4ea52, 92b8396827f24c6887da469b9b816895, 9d6b390dce8b4ab5b978ea2286e63554; no voters:
I20260807 19:42:47.981913 7969 raft_consensus.cc:2781] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:47.982030 7969 raft_consensus.cc:482] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:47.982141 7969 raft_consensus.cc:3037] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:47.985481 7969 raft_consensus.cc:504] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.986742 7969 leader_election.cc:288] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [CANDIDATE]: Term 1 election: Requested vote from peers 92b8396827f24c6887da469b9b816895 (127.30.208.1:37733), 9d6b390dce8b4ab5b978ea2286e63554 (127.30.208.2:38301)
I20260807 19:42:47.986878 7682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6b3c17a0814ae0b155b6fdfcc9799d" candidate_uuid: "2677256de76840bfbd379bcb63c4ea52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d6b390dce8b4ab5b978ea2286e63554"
I20260807 19:42:47.987138 7682 raft_consensus.cc:3037] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:47.987138 7524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c6b3c17a0814ae0b155b6fdfcc9799d" candidate_uuid: "2677256de76840bfbd379bcb63c4ea52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92b8396827f24c6887da469b9b816895"
I20260807 19:42:47.987361 7524 raft_consensus.cc:3037] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:47.990259 7682 raft_consensus.cc:2454] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2677256de76840bfbd379bcb63c4ea52 in term 1.
I20260807 19:42:47.990495 7524 raft_consensus.cc:2454] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2677256de76840bfbd379bcb63c4ea52 in term 1.
I20260807 19:42:47.990870 7758 leader_election.cc:302] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2677256de76840bfbd379bcb63c4ea52, 9d6b390dce8b4ab5b978ea2286e63554; no voters:
I20260807 19:42:47.991442 7969 raft_consensus.cc:2781] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:47.991986 7969 raft_consensus.cc:686] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [term 1 LEADER]: Becoming Leader. State: Replica: 2677256de76840bfbd379bcb63c4ea52, State: Running, Role: LEADER
I20260807 19:42:47.992280 7969 consensus_queue.cc:237] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:47.995779 7348 catalog_manager.cc:5495] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2677256de76840bfbd379bcb63c4ea52 (127.30.208.3). New cstate: current_term: 1 leader_uuid: "2677256de76840bfbd379bcb63c4ea52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } health_report { overall_health: HEALTHY } } }
I20260807 19:42:48.040289 7915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:48.033126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50980 (local address 127.30.208.3:43607)
0807 19:42:48.033381 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:48.033387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:48.033656 (+ 269us) server_negotiation.cc:407] Connection header received
0807 19:42:48.033742 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:48.033748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:48.033835 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:48.033912 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:48.034488 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:48.035646 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:48.037643 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:48.038326 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:48.038440 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:48.039030 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:48.039243 (+ 213us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:48.039356 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:48.039894 (+ 538us) server_negotiation.cc:299] Negotiation successful
0807 19:42:48.039988 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260807 19:42:48.044411 7682 raft_consensus.cc:1261] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Refusing update from remote peer 2677256de76840bfbd379bcb63c4ea52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:48.044682 7524 raft_consensus.cc:1261] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Refusing update from remote peer 2677256de76840bfbd379bcb63c4ea52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:48.045499 7969 consensus_queue.cc:1035] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:48.046042 7969 consensus_queue.cc:1035] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:48.074698 7348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:48.074976 7348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:48.078517 7348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f: Sending DeleteTablet for 3 replicas of tablet 2c6b3c17a0814ae0b155b6fdfcc9799d
I20260807 19:42:48.079489 7501 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c6b3c17a0814ae0b155b6fdfcc9799d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:48 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:38954
I20260807 19:42:48.079512 7812 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c6b3c17a0814ae0b155b6fdfcc9799d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:48 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:50962
I20260807 19:42:48.079788 8003 tablet_replica.cc:330] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52: stopping tablet replica
I20260807 19:42:48.079979 8003 raft_consensus.cc:2229] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:48.079990 8001 tablet_replica.cc:330] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895: stopping tablet replica
I20260807 19:42:48.080204 8001 raft_consensus.cc:2229] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:48.080462 8001 raft_consensus.cc:2258] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:48.080528 8003 raft_consensus.cc:2258] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:48.080860 7655 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c6b3c17a0814ae0b155b6fdfcc9799d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:48 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:57312
I20260807 19:42:48.081403 8002 tablet_replica.cc:330] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554: stopping tablet replica
I20260807 19:42:48.081609 8002 raft_consensus.cc:2229] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:48.081638 8001 ts_tablet_manager.cc:1905] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:48.081854 8002 raft_consensus.cc:2258] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:48.083109 8003 ts_tablet_manager.cc:1905] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:48.088706 8001 ts_tablet_manager.cc:1918] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:48.088887 8001 log.cc:1196] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/2c6b3c17a0814ae0b155b6fdfcc9799d
I20260807 19:42:48.089363 8001 ts_tablet_manager.cc:1939] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 92b8396827f24c6887da469b9b816895: Deleting consensus metadata
I20260807 19:42:48.090531 7330 catalog_manager.cc:4841] TS 92b8396827f24c6887da469b9b816895 (127.30.208.1:37733): tablet 2c6b3c17a0814ae0b155b6fdfcc9799d (table test-table [id=08db99312a814d2a86f606b49d6be646]) successfully deleted
I20260807 19:42:48.091013 7349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:42:48.091795 7349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:42:48.091812 8003 ts_tablet_manager.cc:1918] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:48.092294 8003 log.cc:1196] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/2c6b3c17a0814ae0b155b6fdfcc9799d
I20260807 19:42:48.092782 8003 ts_tablet_manager.cc:1939] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 2677256de76840bfbd379bcb63c4ea52: Deleting consensus metadata
I20260807 19:42:48.094009 7331 catalog_manager.cc:4841] TS 2677256de76840bfbd379bcb63c4ea52 (127.30.208.3:43607): tablet 2c6b3c17a0814ae0b155b6fdfcc9799d (table test-table [id=08db99312a814d2a86f606b49d6be646]) successfully deleted
I20260807 19:42:48.094937 8002 ts_tablet_manager.cc:1905] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:48.100893 8002 ts_tablet_manager.cc:1918] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:42:48.101071 8002 log.cc:1196] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/2c6b3c17a0814ae0b155b6fdfcc9799d
I20260807 19:42:48.102118 8002 ts_tablet_manager.cc:1939] T 2c6b3c17a0814ae0b155b6fdfcc9799d P 9d6b390dce8b4ab5b978ea2286e63554: Deleting consensus metadata
I20260807 19:42:48.103627 7501 tablet_service.cc:1467] Processing CreateTablet for tablet d61c8cea5ef44c64a55b8e9268fc5ba1 (DEFAULT_TABLE table=test-table [id=f056a56d942446cba3578f9680013e9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:48.104099 7501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d61c8cea5ef44c64a55b8e9268fc5ba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:48.104009 7812 tablet_service.cc:1467] Processing CreateTablet for tablet d61c8cea5ef44c64a55b8e9268fc5ba1 (DEFAULT_TABLE table=test-table [id=f056a56d942446cba3578f9680013e9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:48.104449 7812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d61c8cea5ef44c64a55b8e9268fc5ba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:48.110028 7655 tablet_service.cc:1467] Processing CreateTablet for tablet d61c8cea5ef44c64a55b8e9268fc5ba1 (DEFAULT_TABLE table=test-table [id=f056a56d942446cba3578f9680013e9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:42:48.110527 7655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d61c8cea5ef44c64a55b8e9268fc5ba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:48.110808 7333 catalog_manager.cc:4841] TS 9d6b390dce8b4ab5b978ea2286e63554 (127.30.208.2:38301): tablet 2c6b3c17a0814ae0b155b6fdfcc9799d (table test-table [id=08db99312a814d2a86f606b49d6be646]) successfully deleted
I20260807 19:42:48.113016 8018 tablet_bootstrap.cc:492] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52: Bootstrap starting.
I20260807 19:42:48.114846 8018 tablet_bootstrap.cc:654] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:48.116468 8017 tablet_bootstrap.cc:492] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895: Bootstrap starting.
I20260807 19:42:48.116680 8019 tablet_bootstrap.cc:492] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554: Bootstrap starting.
I20260807 19:42:48.118218 8017 tablet_bootstrap.cc:654] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:48.118247 8019 tablet_bootstrap.cc:654] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:48.120078 8018 tablet_bootstrap.cc:492] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52: No bootstrap required, opened a new log
I20260807 19:42:48.120173 8017 tablet_bootstrap.cc:492] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895: No bootstrap required, opened a new log
I20260807 19:42:48.120265 8018 ts_tablet_manager.cc:1397] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 19:42:48.120365 8017 ts_tablet_manager.cc:1397] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 19:42:48.121052 8018 raft_consensus.cc:348] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:48.121362 8018 raft_consensus.cc:374] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:48.121446 8018 raft_consensus.cc:729] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2677256de76840bfbd379bcb63c4ea52, State: Initialized, Role: FOLLOWER
I20260807 19:42:48.122017 8018 consensus_queue.cc:260] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:48.121145 8017 raft_consensus.cc:348] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:48.122517 8017 raft_consensus.cc:374] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:48.122618 8017 raft_consensus.cc:729] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92b8396827f24c6887da469b9b816895, State: Initialized, Role: FOLLOWER
I20260807 19:42:48.123135 8018 ts_tablet_manager.cc:1428] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 19:42:48.127861 8017 consensus_queue.cc:260] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:48.130899 8017 ts_tablet_manager.cc:1428] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260807 19:42:48.135739 8019 tablet_bootstrap.cc:492] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554: No bootstrap required, opened a new log
I20260807 19:42:48.135934 8019 ts_tablet_manager.cc:1397] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.008s
I20260807 19:42:48.136778 8019 raft_consensus.cc:348] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:48.137064 8019 raft_consensus.cc:374] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:48.137148 8019 raft_consensus.cc:729] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d6b390dce8b4ab5b978ea2286e63554, State: Initialized, Role: FOLLOWER
I20260807 19:42:48.137461 8019 consensus_queue.cc:260] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:48.138047 8019 ts_tablet_manager.cc:1428] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 19:42:48.409915 7923 raft_consensus.cc:482] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:42:48.410167 7923 raft_consensus.cc:504] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:48.411046 7923 leader_election.cc:288] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d6b390dce8b4ab5b978ea2286e63554 (127.30.208.2:38301), 2677256de76840bfbd379bcb63c4ea52 (127.30.208.3:43607)
I20260807 19:42:48.411579 7681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61c8cea5ef44c64a55b8e9268fc5ba1" candidate_uuid: "92b8396827f24c6887da469b9b816895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d6b390dce8b4ab5b978ea2286e63554" is_pre_election: true
I20260807 19:42:48.411618 7837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61c8cea5ef44c64a55b8e9268fc5ba1" candidate_uuid: "92b8396827f24c6887da469b9b816895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2677256de76840bfbd379bcb63c4ea52" is_pre_election: true
I20260807 19:42:48.411818 7681 raft_consensus.cc:2454] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92b8396827f24c6887da469b9b816895 in term 0.
I20260807 19:42:48.411890 7837 raft_consensus.cc:2454] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92b8396827f24c6887da469b9b816895 in term 0.
I20260807 19:42:48.412386 7449 leader_election.cc:302] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92b8396827f24c6887da469b9b816895, 9d6b390dce8b4ab5b978ea2286e63554; no voters:
I20260807 19:42:48.412701 7923 raft_consensus.cc:2781] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:42:48.412819 7923 raft_consensus.cc:482] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:42:48.412917 7923 raft_consensus.cc:3037] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:48.415047 7923 raft_consensus.cc:504] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:48.415588 7923 leader_election.cc:288] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [CANDIDATE]: Term 1 election: Requested vote from peers 9d6b390dce8b4ab5b978ea2286e63554 (127.30.208.2:38301), 2677256de76840bfbd379bcb63c4ea52 (127.30.208.3:43607)
I20260807 19:42:48.416160 7837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61c8cea5ef44c64a55b8e9268fc5ba1" candidate_uuid: "92b8396827f24c6887da469b9b816895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2677256de76840bfbd379bcb63c4ea52"
I20260807 19:42:48.416167 7681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61c8cea5ef44c64a55b8e9268fc5ba1" candidate_uuid: "92b8396827f24c6887da469b9b816895" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d6b390dce8b4ab5b978ea2286e63554"
I20260807 19:42:48.416356 7837 raft_consensus.cc:3037] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:48.416400 7681 raft_consensus.cc:3037] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:48.418711 7837 raft_consensus.cc:2454] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92b8396827f24c6887da469b9b816895 in term 1.
I20260807 19:42:48.419135 7448 leader_election.cc:302] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2677256de76840bfbd379bcb63c4ea52, 92b8396827f24c6887da469b9b816895; no voters:
I20260807 19:42:48.419409 7923 raft_consensus.cc:2781] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:48.419448 7681 raft_consensus.cc:2454] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92b8396827f24c6887da469b9b816895 in term 1.
I20260807 19:42:48.419822 7923 raft_consensus.cc:686] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [term 1 LEADER]: Becoming Leader. State: Replica: 92b8396827f24c6887da469b9b816895, State: Running, Role: LEADER
I20260807 19:42:48.420130 7923 consensus_queue.cc:237] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } }
I20260807 19:42:48.422979 7349 catalog_manager.cc:5495] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92b8396827f24c6887da469b9b816895 (127.30.208.1). New cstate: current_term: 1 leader_uuid: "92b8396827f24c6887da469b9b816895" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92b8396827f24c6887da469b9b816895" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 37733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 } health_report { overall_health: UNKNOWN } } }
I20260807 19:42:48.496917 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:48.491589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38968 (local address 127.30.208.1:37733)
0807 19:42:48.492090 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:48.492094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:48.492109 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:42:48.492161 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:48.492164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:48.492228 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:48.492314 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:42:48.492719 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:48.493544 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:48.495164 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:48.495606 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:48.495721 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:48.496171 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:42:48.496292 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:42:48.496369 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:48.496513 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 19:42:48.496598 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":244,"threads_started":1}
I20260807 19:42:48.507254 7681 raft_consensus.cc:1261] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Refusing update from remote peer 92b8396827f24c6887da469b9b816895: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:48.507254 7837 raft_consensus.cc:1261] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Refusing update from remote peer 92b8396827f24c6887da469b9b816895: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:42:48.507893 7923 consensus_queue.cc:1035] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2677256de76840bfbd379bcb63c4ea52" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:48.508165 8032 consensus_queue.cc:1035] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d6b390dce8b4ab5b978ea2286e63554" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 38301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:42:48.576452 7349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:48.576762 7349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:42:48.579667 7349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f32b18cc46444585af278750d8ceef8f: Sending DeleteTablet for 3 replicas of tablet d61c8cea5ef44c64a55b8e9268fc5ba1
I20260807 19:42:48.580524 7655 tablet_service.cc:1514] Processing DeleteTablet for tablet d61c8cea5ef44c64a55b8e9268fc5ba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:48 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:57312
I20260807 19:42:48.580536 7501 tablet_service.cc:1514] Processing DeleteTablet for tablet d61c8cea5ef44c64a55b8e9268fc5ba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:48 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:38954
I20260807 19:42:48.580852 8001 tablet_replica.cc:330] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895: stopping tablet replica
I20260807 19:42:48.580854 8002 tablet_replica.cc:330] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554: stopping tablet replica
I20260807 19:42:48.581048 8001 raft_consensus.cc:2229] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:42:48.581048 8002 raft_consensus.cc:2229] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:48.581315 8002 raft_consensus.cc:2258] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:48.581472 8001 raft_consensus.cc:2258] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:48.581787 7812 tablet_service.cc:1514] Processing DeleteTablet for tablet d61c8cea5ef44c64a55b8e9268fc5ba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:42:48 UTC) from {username='oryx', principal='oryx/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:50962
I20260807 19:42:48.582177 8003 tablet_replica.cc:330] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52: stopping tablet replica
I20260807 19:42:48.582373 8003 raft_consensus.cc:2229] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:42:48.582613 8003 raft_consensus.cc:2258] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:42:48.582787 8001 ts_tablet_manager.cc:1905] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:48.583379 8002 ts_tablet_manager.cc:1905] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:48.586002 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 7430
I20260807 19:42:48.586017 8003 ts_tablet_manager.cc:1905] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:42:48.590368 8001 ts_tablet_manager.cc:1918] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:42:48.590497 8001 log.cc:1196] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/d61c8cea5ef44c64a55b8e9268fc5ba1
I20260807 19:42:48.590883 8001 ts_tablet_manager.cc:1939] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 92b8396827f24c6887da469b9b816895: Deleting consensus metadata
I20260807 19:42:48.590911 8002 ts_tablet_manager.cc:1918] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:42:48.591018 8002 log.cc:1196] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/d61c8cea5ef44c64a55b8e9268fc5ba1
I20260807 19:42:48.591668 8002 ts_tablet_manager.cc:1939] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 9d6b390dce8b4ab5b978ea2286e63554: Deleting consensus metadata
I20260807 19:42:48.592113 7330 catalog_manager.cc:4841] TS 92b8396827f24c6887da469b9b816895 (127.30.208.1:37733): tablet d61c8cea5ef44c64a55b8e9268fc5ba1 (table test-table [id=f056a56d942446cba3578f9680013e9a]) successfully deleted
I20260807 19:42:48.592834 7333 catalog_manager.cc:4841] TS 9d6b390dce8b4ab5b978ea2286e63554 (127.30.208.2:38301): tablet d61c8cea5ef44c64a55b8e9268fc5ba1 (table test-table [id=f056a56d942446cba3578f9680013e9a]) successfully deleted
I20260807 19:42:48.593519 8003 ts_tablet_manager.cc:1918] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 19:42:48.593642 8003 log.cc:1196] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/d61c8cea5ef44c64a55b8e9268fc5ba1
I20260807 19:42:48.594012 8003 ts_tablet_manager.cc:1939] T d61c8cea5ef44c64a55b8e9268fc5ba1 P 2677256de76840bfbd379bcb63c4ea52: Deleting consensus metadata
I20260807 19:42:48.594854 7331 catalog_manager.cc:4841] TS 2677256de76840bfbd379bcb63c4ea52 (127.30.208.3:43607): tablet d61c8cea5ef44c64a55b8e9268fc5ba1 (table test-table [id=f056a56d942446cba3578f9680013e9a]) successfully deleted
Aug 07 19:42:48 dist-test-slave-vq2q krb5kdc[7299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.208.1@KRBTEST.COM
I20260807 19:42:48.606271 8035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:48.587220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38976 (local address 127.30.208.1:37733)
0807 19:42:48.588956 (+ 1736us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:48.588963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:48.593056 (+ 4093us) server_negotiation.cc:407] Connection header received
0807 19:42:48.593290 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:48.593295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:48.593357 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:48.593438 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:48.594707 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:48.595546 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:48.596730 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:48.597125 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:48.597275 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:48.601173 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:48.601222 (+ 49us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:48.601245 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:48.601283 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:48.603860 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:48.604530 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:48.604536 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:48.604539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:48.604667 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:48.604951 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:48.604953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:48.604955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:48.605398 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:48.605646 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:48.605845 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 19:42:48.606018 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1606}
I20260807 19:42:48.606921 7571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:48.739903 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 7430
I20260807 19:42:48.772421 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 7580
Aug 07 19:42:48 dist-test-slave-vq2q krb5kdc[7299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.208.2@KRBTEST.COM
I20260807 19:42:48.787335 8044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:48.773160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57330 (local address 127.30.208.2:38301)
0807 19:42:48.773616 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:48.773621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:48.773637 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:42:48.773691 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:48.773695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:48.773769 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:48.773959 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:48.775142 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:48.776171 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:48.777594 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:48.777987 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:48.778086 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:48.782025 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:48.782040 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:48.782056 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:48.782090 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:48.784760 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:48.785513 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:48.785520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:48.785523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:48.785675 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:48.785985 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:48.785988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:48.785990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:48.786331 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:48.786557 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:48.786805 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 19:42:48.786993 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":216,"threads_started":1}
I20260807 19:42:48.788007 7726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:48.911523 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 7580
I20260807 19:42:48.934480 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 7737
Aug 07 19:42:48 dist-test-slave-vq2q krb5kdc[7299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.208.3@KRBTEST.COM
I20260807 19:42:48.950506 8046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:48.935252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.30.208.3:43607)
0807 19:42:48.935727 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:48.935731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:48.935748 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:42:48.935802 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:48.935806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:48.935896 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:48.935986 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:48.937272 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:48.937989 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:48.939326 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:48.939772 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:48.939882 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:48.944633 (+ 4751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:48.944652 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:48.944668 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:48.944709 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:48.947558 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:48.948301 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:48.948328 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:48.948330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:48.948480 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:48.948940 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:48.948945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:48.948947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:48.949433 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:48.949698 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:48.949977 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 19:42:48.950165 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":229,"threads_started":1}
I20260807 19:42:48.951190 7883 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:49.107566 8050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:49.106169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:37733 (local address 127.30.208.3:36283)
0807 19:42:49.107152 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:49.107315 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:37733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":870,"thread_start_us":966,"threads_started":1}
I20260807 19:42:49.107573 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 7737
I20260807 19:42:49.132658 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 7313
I20260807 19:42:49.133936 7394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:49.252147 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 7313
2026-08-07T19:42:49Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7029 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:49 dist-test-slave-vq2q krb5kdc[8054](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:49 dist-test-slave-vq2q krb5kdc[8054](info): set up 2 sockets
Aug 07 19:42:49 dist-test-slave-vq2q krb5kdc[8054](info): commencing operation
krb5kdc: starting...
W20260807 19:42:50.344148 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:42:50 dist-test-slave-vq2q krb5kdc[8054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:42:50.601513 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:38767
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:35065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:38767
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:50.942054 8069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:50.942493 8069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:50.942642 8069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:50.953665 8069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:50.953799 8069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:50.953881 8069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:50.953970 8069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:50.954038 8069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:50.964229 8069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:35065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:38767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:38767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:50.965636 8069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:50.966938 8069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:50.978330 8075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:50.978371 8076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:50.978396 8079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:50.979053 8069 server_base.cc:1034] running on GCE node
I20260807 19:42:50.979943 8069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:50.981408 8069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:50.982640 8069 hybrid_clock.cc:648] HybridClock initialized: now 1786131770982592 us; error 38 us; skew 500 ppm
I20260807 19:42:50.983017 8069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T19:42:51Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1800 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:42:51 dist-test-slave-vq2q krb5kdc[8085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:42:51 dist-test-slave-vq2q krb5kdc[8085](info): set up 2 sockets
Aug 07 19:42:51 dist-test-slave-vq2q krb5kdc[8085](info): commencing operation
krb5kdc: starting...
W20260807 19:42:52.132390 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.003s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:42:52 dist-test-slave-vq2q krb5kdc[8085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:42:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:42:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:42:52.392449 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:39925
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:39925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:52.719475 8100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:52.719762 8100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:52.719885 8100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:52.727100 8100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:52.727207 8100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:52.727264 8100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:52.727348 8100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:52.727401 8100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:52.737277 8100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:39925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:39925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:52.738569 8100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:52.739850 8100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:52.751081 8106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:52.751055 8108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:52.751396 8105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:52.751782 8100 server_base.cc:1034] running on GCE node
I20260807 19:42:52.752605 8100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:52.754125 8100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:52.755354 8100 hybrid_clock.cc:648] HybridClock initialized: now 1786131772755299 us; error 46 us; skew 500 ppm
I20260807 19:42:52.755708 8100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:52 dist-test-slave-vq2q krb5kdc[8085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131772, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:52.762584 8100 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:42:52.764091 8100 webserver.cc:460] Webserver started at http://127.30.208.62:39463/ using document root <none> and password file <none>
I20260807 19:42:52.764757 8100 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:52.764891 8100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:52.765209 8100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:52.768201 8100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "6a0357af8f404d6e9b67391dc41f4834"
format_stamp: "Formatted at 2026-08-07 19:42:52 on dist-test-slave-vq2q"
server_key: "3eee6369ec6733c59efe5b5473cc0908"
server_key_iv: "b0c6226bb72063c83f4e6fe0ac169771"
server_key_version: "encryptionkey@0"
I20260807 19:42:52.768945 8100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "6a0357af8f404d6e9b67391dc41f4834"
format_stamp: "Formatted at 2026-08-07 19:42:52 on dist-test-slave-vq2q"
server_key: "3eee6369ec6733c59efe5b5473cc0908"
server_key_iv: "b0c6226bb72063c83f4e6fe0ac169771"
server_key_version: "encryptionkey@0"
I20260807 19:42:52.774094 8100 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260807 19:42:52.777554 8116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:52.778898 8100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:42:52.779062 8100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "6a0357af8f404d6e9b67391dc41f4834"
format_stamp: "Formatted at 2026-08-07 19:42:52 on dist-test-slave-vq2q"
server_key: "3eee6369ec6733c59efe5b5473cc0908"
server_key_iv: "b0c6226bb72063c83f4e6fe0ac169771"
server_key_version: "encryptionkey@0"
I20260807 19:42:52.779253 8100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:52.795849 8100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:52.799998 8100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:52.800251 8100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:52.816524 8100 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:39925
I20260807 19:42:52.816530 8172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:39925 every 8 connection(s)
I20260807 19:42:52.817970 8100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:52.821710 8173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:42:52.827808 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 8100
I20260807 19:42:52.828020 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:42:52.828644 31552 external_mini_cluster.cc:1428] Setting key 14c44943c64d19efb4d4717e59e62322
I20260807 19:42:52.832537 8173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834: Bootstrap starting.
I20260807 19:42:52.836352 8173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834: Neither blocks nor log segments found. Creating new log.
I20260807 19:42:52.837528 8173 log.cc:824] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:52.840929 8173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834: No bootstrap required, opened a new log
Aug 07 19:42:52 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:52.849258 8173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0357af8f404d6e9b67391dc41f4834" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 39925 } }
I20260807 19:42:52.849658 8173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:42:52.849768 8173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a0357af8f404d6e9b67391dc41f4834, State: Initialized, Role: FOLLOWER
I20260807 19:42:52.850427 8173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0357af8f404d6e9b67391dc41f4834" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 39925 } }
I20260807 19:42:52.850677 8173 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:52.850872 8173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:52.851074 8173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:42:52.852159 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:52.830770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41782 (local address 127.30.208.62:39925)
0807 19:42:52.831579 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:52.831591 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:52.831633 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:42:52.833496 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:52.833520 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:52.833897 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:52.834254 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:52.836166 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:52.837670 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:52.839055 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:52.839750 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:52.839888 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:52.844579 (+ 4691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:52.844587 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:52.844607 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:52.844650 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:52.847781 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:52.848554 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:52.848561 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:52.848569 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:52.848751 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:52.849135 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:52.849139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:52.849141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:52.849828 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:52.850122 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:52.850569 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 19:42:52.851038 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":355,"threads_started":1}
I20260807 19:42:52.853595 8173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0357af8f404d6e9b67391dc41f4834" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 39925 } }
I20260807 19:42:52.854171 8173 leader_election.cc:302] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a0357af8f404d6e9b67391dc41f4834; no voters:
I20260807 19:42:52.854904 8173 leader_election.cc:288] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:42:52.855048 8179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:42:52.855756 8179 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 1 LEADER]: Becoming Leader. State: Replica: 6a0357af8f404d6e9b67391dc41f4834, State: Running, Role: LEADER
I20260807 19:42:52.856287 8179 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0357af8f404d6e9b67391dc41f4834" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 39925 } }
I20260807 19:42:52.856957 8173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:52.858892 8182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a0357af8f404d6e9b67391dc41f4834. Latest consensus state: current_term: 1 leader_uuid: "6a0357af8f404d6e9b67391dc41f4834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0357af8f404d6e9b67391dc41f4834" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 39925 } } }
I20260807 19:42:52.858883 8180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a0357af8f404d6e9b67391dc41f4834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0357af8f404d6e9b67391dc41f4834" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 39925 } } }
I20260807 19:42:52.859314 8182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:52.859459 8180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:52.861050 8185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:52.866724 8185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:52.876560 8185 catalog_manager.cc:1347] Generated new cluster ID: e4f622910fb0426b8392c033e8f53a10
I20260807 19:42:52.876694 8185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:52.884326 8185 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:42:52.885639 8185 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:52.897775 8185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834: Generated new TSK 0
I20260807 19:42:52.898641 8185 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:42:52.960980 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:39925
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:53.296660 8202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:53.296948 8202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:53.297256 8202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:53.305109 8202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:53.305270 8202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:53.305442 8202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:42:53.315645 8202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:39925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:53.317042 8202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:53.318385 8202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:53.331118 8212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:53.331143 8210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:53.331351 8209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:53.331954 8202 server_base.cc:1034] running on GCE node
I20260807 19:42:53.332525 8202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:53.333488 8202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:53.334685 8202 hybrid_clock.cc:648] HybridClock initialized: now 1786131773334637 us; error 42 us; skew 500 ppm
I20260807 19:42:53.334986 8202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:53 dist-test-slave-vq2q krb5kdc[8085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131773, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:53.338032 8202 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:42:53.339411 8202 webserver.cc:460] Webserver started at http://127.30.208.1:33625/ using document root <none> and password file <none>
I20260807 19:42:53.340121 8202 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:53.340224 8202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:53.340483 8202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:53.343366 8202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "a47c7d5502a8424eb5e2288ac10e614a"
format_stamp: "Formatted at 2026-08-07 19:42:53 on dist-test-slave-vq2q"
server_key: "1948c45768e5e6ad0a4547583d649d5c"
server_key_iv: "830c3e489543c9c632b35a6f79c7e5f1"
server_key_version: "encryptionkey@0"
I20260807 19:42:53.344125 8202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "a47c7d5502a8424eb5e2288ac10e614a"
format_stamp: "Formatted at 2026-08-07 19:42:53 on dist-test-slave-vq2q"
server_key: "1948c45768e5e6ad0a4547583d649d5c"
server_key_iv: "830c3e489543c9c632b35a6f79c7e5f1"
server_key_version: "encryptionkey@0"
I20260807 19:42:53.349359 8202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 19:42:53.353583 8219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:53.355053 8202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:53.355219 8202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "a47c7d5502a8424eb5e2288ac10e614a"
format_stamp: "Formatted at 2026-08-07 19:42:53 on dist-test-slave-vq2q"
server_key: "1948c45768e5e6ad0a4547583d649d5c"
server_key_iv: "830c3e489543c9c632b35a6f79c7e5f1"
server_key_version: "encryptionkey@0"
I20260807 19:42:53.355437 8202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:53.394202 8202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:53.398788 8202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:53.399091 8202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:53.400117 8202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:53.401576 8202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:53.401660 8202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:53.401773 8202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:53.401824 8202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:53.431449 8202 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:40541
I20260807 19:42:53.431496 8344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:40541 every 8 connection(s)
I20260807 19:42:53.432705 8202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:53.438537 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 8202
I20260807 19:42:53.438764 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:42:53.439551 31552 external_mini_cluster.cc:1428] Setting key 3362ee7d42cfcc87206f6d72174eb776
Aug 07 19:42:53 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131773, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:53.453570 8349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:53.435241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:54337 (local address 127.30.208.62:39925)
0807 19:42:53.435723 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:53.435728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:53.436636 (+ 908us) server_negotiation.cc:407] Connection header received
0807 19:42:53.438650 (+ 2014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:53.438656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:53.438740 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:53.438847 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:53.440643 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:53.441510 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:53.443626 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:53.444010 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:53.444101 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:53.447955 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:53.447960 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:53.447963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:53.447994 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:53.450415 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:53.451206 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:53.451209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:53.451212 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:53.451335 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:53.451784 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:53.451787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:53.451789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:53.452030 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:53.452138 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:53.452963 (+ 825us) server_negotiation.cc:299] Negotiation successful
0807 19:42:53.453205 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":251,"threads_started":1}
I20260807 19:42:53.454344 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:53.435763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:54337)
0807 19:42:53.436460 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:53.436504 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:53.438343 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:53.439114 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:53.439125 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:53.439857 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:53.440501 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:53.440519 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:53.441654 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:53.441659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:53.443505 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:53.443520 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:53.444219 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:53.444227 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:53.444495 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:53.445198 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:53.445230 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:53.447735 (+ 2505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:53.450548 (+ 2813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:53.450556 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:53.450572 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:53.451111 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:53.451438 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:53.451444 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:53.451447 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:53.451693 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:53.452139 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:53.452147 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:53.452570 (+ 423us) client_negotiation.cc:769] Sending connection context
0807 19:42:53.452833 (+ 263us) client_negotiation.cc:240] Negotiation successful
0807 19:42:53.453271 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":259,"threads_started":1}
I20260807 19:42:53.455859 8345 heartbeater.cc:347] Connected to a master server at 127.30.208.62:39925
I20260807 19:42:53.456190 8345 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:53.457047 8345 heartbeater.cc:511] Master 127.30.208.62:39925 requested a full tablet report, sending...
I20260807 19:42:53.459321 8135 ts_manager.cc:194] Registered new tserver with Master: a47c7d5502a8424eb5e2288ac10e614a (127.30.208.1:40541)
I20260807 19:42:53.460811 8135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:54337
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:42:53.497027 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:39925
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:53.837056 8353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:53.837488 8353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:53.837642 8353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:53.845997 8353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:53.846113 8353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:53.846227 8353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:42:53.856416 8353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:39925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:53.857848 8353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:53.859210 8353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:53.872176 8359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:53.872702 8362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:53.872752 8364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:53.873059 8353 server_base.cc:1034] running on GCE node
I20260807 19:42:53.873629 8353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:53.874507 8353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:53.875746 8353 hybrid_clock.cc:648] HybridClock initialized: now 1786131773875667 us; error 42 us; skew 500 ppm
I20260807 19:42:53.876034 8353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:53 dist-test-slave-vq2q krb5kdc[8085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131773, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:53.879127 8353 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:42:53.880554 8353 webserver.cc:460] Webserver started at http://127.30.208.2:36599/ using document root <none> and password file <none>
I20260807 19:42:53.881305 8353 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:53.881426 8353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:53.881727 8353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:53.884864 8353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "af8de40424814d89acbd808daa1c9d97"
format_stamp: "Formatted at 2026-08-07 19:42:53 on dist-test-slave-vq2q"
server_key: "e23e19183e6b4a344ee734bd588ae6b9"
server_key_iv: "d7ebdc16ca63d8a366b478313fcc84c2"
server_key_version: "encryptionkey@0"
I20260807 19:42:53.885715 8353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "af8de40424814d89acbd808daa1c9d97"
format_stamp: "Formatted at 2026-08-07 19:42:53 on dist-test-slave-vq2q"
server_key: "e23e19183e6b4a344ee734bd588ae6b9"
server_key_iv: "d7ebdc16ca63d8a366b478313fcc84c2"
server_key_version: "encryptionkey@0"
I20260807 19:42:53.891000 8353 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:42:53.894552 8371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:53.895961 8353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:53.896126 8353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "af8de40424814d89acbd808daa1c9d97"
format_stamp: "Formatted at 2026-08-07 19:42:53 on dist-test-slave-vq2q"
server_key: "e23e19183e6b4a344ee734bd588ae6b9"
server_key_iv: "d7ebdc16ca63d8a366b478313fcc84c2"
server_key_version: "encryptionkey@0"
I20260807 19:42:53.896323 8353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:53.912990 8353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:53.917704 8353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:53.918001 8353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:53.919057 8353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:53.920434 8353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:53.920514 8353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:53.920642 8353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:53.920706 8353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:53.951360 8353 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:35113
I20260807 19:42:53.951402 8494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:35113 every 8 connection(s)
I20260807 19:42:53.952633 8353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:53.962131 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 8353
I20260807 19:42:53.962339 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:42:53.962857 31552 external_mini_cluster.cc:1428] Setting key c81433321441601e64cd1e9772a0cc93
Aug 07 19:42:53 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131773, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:53.973440 8498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:53.955240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:57927 (local address 127.30.208.62:39925)
0807 19:42:53.955681 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:53.955688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:53.956643 (+ 955us) server_negotiation.cc:407] Connection header received
0807 19:42:53.958473 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:53.958477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:53.958545 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:53.958630 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:53.960102 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:53.960920 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:53.962882 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:53.963427 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:53.963562 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:53.967659 (+ 4097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:53.967665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:53.967668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:53.967706 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:53.970263 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:53.971103 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:53.971107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:53.971110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:53.971269 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:53.971717 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:53.971720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:53.971722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:53.971989 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:53.972101 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:53.972937 (+ 836us) server_negotiation.cc:299] Negotiation successful
0807 19:42:53.973123 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":201,"threads_started":1}
I20260807 19:42:53.974673 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:53.955810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:57927)
0807 19:42:53.956488 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:53.956525 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:53.958225 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:53.958806 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:53.958832 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:53.959381 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:53.959918 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:53.959936 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:53.961085 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:53.961089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:53.962706 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:53.962722 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:53.963654 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:53.963662 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:53.963967 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:53.964652 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:53.964683 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:53.967411 (+ 2728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:53.970411 (+ 3000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:53.970420 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:53.970436 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:53.970977 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:53.971373 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:53.971377 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:53.971380 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:53.971621 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:53.972105 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:53.972113 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:53.972532 (+ 419us) client_negotiation.cc:769] Sending connection context
0807 19:42:53.972823 (+ 291us) client_negotiation.cc:240] Negotiation successful
0807 19:42:53.973366 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":253,"threads_started":1}
I20260807 19:42:53.976238 8496 heartbeater.cc:347] Connected to a master server at 127.30.208.62:39925
I20260807 19:42:53.976534 8496 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:53.977308 8496 heartbeater.cc:511] Master 127.30.208.62:39925 requested a full tablet report, sending...
I20260807 19:42:53.978791 8135 ts_manager.cc:194] Registered new tserver with Master: af8de40424814d89acbd808daa1c9d97 (127.30.208.2:35113)
I20260807 19:42:53.979592 8135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:57927
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:42:54.020896 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:39925
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:54.350849 8504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:54.351145 8504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:54.351292 8504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:54.358644 8504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:54.358770 8504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:54.358918 8504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:42:54.368808 8504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:39925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:54.370142 8504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:54.371389 8504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:54.384460 8514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:54.384487 8512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:54.384433 8511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:54.385231 8504 server_base.cc:1034] running on GCE node
I20260807 19:42:54.385946 8504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:54.386927 8504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:54.388164 8504 hybrid_clock.cc:648] HybridClock initialized: now 1786131774388096 us; error 58 us; skew 500 ppm
I20260807 19:42:54.388514 8504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:54 dist-test-slave-vq2q krb5kdc[8085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131774, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:54.391863 8504 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:42:54.393468 8504 webserver.cc:460] Webserver started at http://127.30.208.3:43609/ using document root <none> and password file <none>
I20260807 19:42:54.394176 8504 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:54.394280 8504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:54.394553 8504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:42:54.397595 8504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "9ad67a2f99fc4544af4c7654e38a3683"
format_stamp: "Formatted at 2026-08-07 19:42:54 on dist-test-slave-vq2q"
server_key: "6075951608454a05827b9239c028676d"
server_key_iv: "c6d44662f09fb1b22d83a650accb60c2"
server_key_version: "encryptionkey@0"
I20260807 19:42:54.398402 8504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "9ad67a2f99fc4544af4c7654e38a3683"
format_stamp: "Formatted at 2026-08-07 19:42:54 on dist-test-slave-vq2q"
server_key: "6075951608454a05827b9239c028676d"
server_key_iv: "c6d44662f09fb1b22d83a650accb60c2"
server_key_version: "encryptionkey@0"
I20260807 19:42:54.403702 8504 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260807 19:42:54.407351 8523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:54.408842 8504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 19:42:54.409008 8504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "9ad67a2f99fc4544af4c7654e38a3683"
format_stamp: "Formatted at 2026-08-07 19:42:54 on dist-test-slave-vq2q"
server_key: "6075951608454a05827b9239c028676d"
server_key_iv: "c6d44662f09fb1b22d83a650accb60c2"
server_key_version: "encryptionkey@0"
I20260807 19:42:54.409276 8504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:54.424556 8504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:54.428915 8504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:54.429231 8504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:54.430271 8504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:54.431715 8504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:54.431798 8504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:54.431905 8504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:54.431957 8504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:54.463955 8504 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:44443
I20260807 19:42:54.464043 8647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:44443 every 8 connection(s)
I20260807 19:42:54.464382 8345 heartbeater.cc:503] Master 127.30.208.62:39925 was elected leader, sending a full tablet report...
I20260807 19:42:54.465420 8504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:54.466499 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 8504
I20260807 19:42:54.466702 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:42:54.467200 31552 external_mini_cluster.cc:1428] Setting key 4a5fbf3c226f602fa851b813ea024d47
Aug 07 19:42:54 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131774, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:42:54.485234 8498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:54.468181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:55373 (local address 127.30.208.62:39925)
0807 19:42:54.468343 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:54.468348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:54.469765 (+ 1417us) server_negotiation.cc:407] Connection header received
0807 19:42:54.471664 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:54.471669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:54.471758 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:54.471841 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:54.473300 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:54.474048 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:54.475574 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:54.475975 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:54.476080 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:54.479452 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:54.479457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:54.479460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:54.479492 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:54.482390 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:54.483124 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:54.483128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:54.483130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:54.483247 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:54.483668 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:54.483671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:54.483673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:54.483975 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:54.484109 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:54.484795 (+ 686us) server_negotiation.cc:299] Negotiation successful
0807 19:42:54.484969 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 19:42:54.486240 8650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:54.468808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:55373)
0807 19:42:54.469592 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:54.469633 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:54.471423 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:54.472078 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:54.472086 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:54.472587 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:54.473117 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:54.473134 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:54.474206 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:54.474210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:54.475438 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:54.475451 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:54.476199 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:54.476207 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:54.476419 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:54.476990 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:54.477011 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:54.479267 (+ 2256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:42:54.482514 (+ 3247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:54.482523 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:54.482539 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:54.483008 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:54.483357 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:42:54.483360 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:42:54.483362 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:42:54.483582 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:42:54.484098 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:42:54.484104 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:42:54.484433 (+ 329us) client_negotiation.cc:769] Sending connection context
0807 19:42:54.484652 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 19:42:54.485075 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":350,"threads_started":1}
I20260807 19:42:54.487737 8648 heartbeater.cc:347] Connected to a master server at 127.30.208.62:39925
I20260807 19:42:54.488034 8648 heartbeater.cc:464] Registering TS with master...
I20260807 19:42:54.488750 8648 heartbeater.cc:511] Master 127.30.208.62:39925 requested a full tablet report, sending...
I20260807 19:42:54.490139 8135 ts_manager.cc:194] Registered new tserver with Master: 9ad67a2f99fc4544af4c7654e38a3683 (127.30.208.3:44443)
I20260807 19:42:54.490934 8135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:55373
I20260807 19:42:54.494838 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.1.keytab.
Entry for principal oryx/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.1.keytab.
WARNING: no policy specified for oryx/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.2.keytab.
Entry for principal oryx/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.2.keytab.
WARNING: no policy specified for oryx/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.3.keytab.
Entry for principal oryx/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.3.keytab.
I20260807 19:42:54.657913 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 8202
Aug 07 19:42:54 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:42:54.677515 8658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:54.659255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45984 (local address 127.30.208.1:40541)
0807 19:42:54.659909 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:54.659917 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:54.659947 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:42:54.660071 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:54.660078 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:54.660292 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:54.660447 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:54.662155 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:54.663553 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:54.665310 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:54.666109 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:54.666257 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:54.670850 (+ 4593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:54.670896 (+ 46us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:54.670913 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:54.670968 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:54.674097 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:54.674960 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:54.674970 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:54.674972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:54.675140 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:54.675686 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:54.675691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:54.675694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:54.676170 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:54.676462 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:54.676907 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 19:42:54.677144 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":327,"threads_started":1}
I20260807 19:42:54.679082 8339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:54.763832 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 8202
I20260807 19:42:54.780411 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 8353
Aug 07 19:42:54 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:42:54.796790 8661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:54.781411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58086 (local address 127.30.208.2:35113)
0807 19:42:54.781927 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:54.781934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:54.781962 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:42:54.782024 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:54.782030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:54.782274 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:54.782456 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:54.783711 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:54.784702 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:54.786062 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:54.786637 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:54.786787 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:54.791085 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:54.791099 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:54.791116 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:54.791155 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:54.794170 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:54.794844 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:54.794850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:54.794853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:54.794974 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:54.795300 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:54.795303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:54.795305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:54.795650 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:54.795899 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:54.796183 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 19:42:54.796419 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":248,"threads_started":1}
I20260807 19:42:54.798203 8489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:54.879943 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 8353
I20260807 19:42:54.896842 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 8504
Aug 07 19:42:54 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:42:54.913158 8663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:54.897839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37410 (local address 127.30.208.3:44443)
0807 19:42:54.898447 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:54.898454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:54.898481 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:42:54.898541 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:54.898547 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:54.898750 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:54.898910 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:54.900249 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:54.901361 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:54.902639 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:54.903212 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:54.903327 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:54.907550 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:54.907562 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:54.907576 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:54.907611 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:54.910496 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:54.911138 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:54.911145 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:54.911148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:54.911283 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:54.911596 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:54.911599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:54.911601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:54.911954 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:54.912187 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:54.912525 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 19:42:54.912769 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":302,"threads_started":1}
I20260807 19:42:54.914609 8643 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:54.997822 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 8504
I20260807 19:42:55.015187 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 8100
I20260807 19:42:55.016481 8167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:42:55.120555 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 8100
I20260807 19:42:55.140892 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:39925
--webserver_interface=127.30.208.62
--webserver_port=39463
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:39925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:42:55.474521 8667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:55.474817 8667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:55.474938 8667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:55.482035 8667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:42:55.482137 8667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:55.482210 8667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:55.482290 8667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:42:55.482347 8667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:42:55.492038 8667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:39925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:39925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=39463
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:55.493382 8667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:55.494688 8667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:55.506186 8672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:55.506259 8675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:55.506263 8673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:55.507218 8667 server_base.cc:1034] running on GCE node
I20260807 19:42:55.508041 8667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:55.509509 8667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:55.510773 8667 hybrid_clock.cc:648] HybridClock initialized: now 1786131775510709 us; error 45 us; skew 500 ppm
I20260807 19:42:55.511130 8667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:55 dist-test-slave-vq2q krb5kdc[8085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131775, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:55.514276 8667 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:42:55.515735 8667 webserver.cc:460] Webserver started at http://127.30.208.62:39463/ using document root <none> and password file <none>
I20260807 19:42:55.516425 8667 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:55.516557 8667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:55.522003 8667 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:42:55.524924 8682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:55.526309 8667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:42:55.526477 8667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "6a0357af8f404d6e9b67391dc41f4834"
format_stamp: "Formatted at 2026-08-07 19:42:52 on dist-test-slave-vq2q"
server_key: "3eee6369ec6733c59efe5b5473cc0908"
server_key_iv: "b0c6226bb72063c83f4e6fe0ac169771"
server_key_version: "encryptionkey@0"
I20260807 19:42:55.527459 8667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:55.546609 8667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:55.550628 8667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:55.550884 8667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:55.568400 8667 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:39925
I20260807 19:42:55.568420 8740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:39925 every 8 connection(s)
I20260807 19:42:55.569940 8667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:42:55.571578 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 8667
I20260807 19:42:55.572319 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:40541
--local_ip_for_outbound_sockets=127.30.208.1
--tserver_master_addrs=127.30.208.62:39925
--webserver_port=33625
--webserver_interface=127.30.208.1
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.208.1
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:42:55.577003 8741 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:42:55.580343 8741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834: Bootstrap starting.
I20260807 19:42:55.599802 8741 log.cc:824] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834: Log is configured to *not* fsync() on all Append() calls
I20260807 19:42:55.608646 8741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:42:55.609248 8741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834: Bootstrap complete.
I20260807 19:42:55.618669 8741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0357af8f404d6e9b67391dc41f4834" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 39925 } }
I20260807 19:42:55.619539 8741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a0357af8f404d6e9b67391dc41f4834, State: Initialized, Role: FOLLOWER
I20260807 19:42:55.620177 8741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0357af8f404d6e9b67391dc41f4834" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 39925 } }
I20260807 19:42:55.620440 8741 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:42:55.620594 8741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:42:55.620765 8741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:42:55.624641 8741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0357af8f404d6e9b67391dc41f4834" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 39925 } }
I20260807 19:42:55.625145 8741 leader_election.cc:302] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a0357af8f404d6e9b67391dc41f4834; no voters:
I20260807 19:42:55.625818 8741 leader_election.cc:288] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:42:55.626037 8745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:42:55.627009 8745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [term 2 LEADER]: Becoming Leader. State: Replica: 6a0357af8f404d6e9b67391dc41f4834, State: Running, Role: LEADER
I20260807 19:42:55.627781 8741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:42:55.627662 8745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0357af8f404d6e9b67391dc41f4834" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 39925 } }
I20260807 19:42:55.630543 8747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a0357af8f404d6e9b67391dc41f4834. Latest consensus state: current_term: 2 leader_uuid: "6a0357af8f404d6e9b67391dc41f4834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0357af8f404d6e9b67391dc41f4834" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 39925 } } }
I20260807 19:42:55.630903 8747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:55.631637 8746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6a0357af8f404d6e9b67391dc41f4834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0357af8f404d6e9b67391dc41f4834" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 39925 } } }
I20260807 19:42:55.631945 8746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834 [sys.catalog]: This master's current role is: LEADER
I20260807 19:42:55.639528 8752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:42:55.646140 8752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:42:55.647605 8752 catalog_manager.cc:1259] Loaded cluster ID: e4f622910fb0426b8392c033e8f53a10
I20260807 19:42:55.647778 8752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:42:55.651876 8752 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:42:55.653436 8752 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6a0357af8f404d6e9b67391dc41f4834: Loaded TSK: 0
I20260807 19:42:55.655059 8752 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:42:55.919057 8743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:55.919343 8743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:55.919459 8743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:55.927166 8743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:55.927299 8743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:55.927448 8743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:42:55.938030 8743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.1.keytab
--principal=oryx/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:40541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=33625
--tserver_master_addrs=127.30.208.62:39925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:55.939491 8743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:55.940932 8743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:55.954811 8771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:55.954900 8772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:55.955135 8774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:55.956573 8743 server_base.cc:1034] running on GCE node
I20260807 19:42:55.957316 8743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:55.958454 8743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:55.959694 8743 hybrid_clock.cc:648] HybridClock initialized: now 1786131775959630 us; error 50 us; skew 500 ppm
I20260807 19:42:55.960053 8743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:55 dist-test-slave-vq2q krb5kdc[8085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131775, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:55.963455 8743 init.cc:377] Logged in from keytab as oryx/127.30.208.1@KRBTEST.COM (short username oryx)
I20260807 19:42:55.965056 8743 webserver.cc:460] Webserver started at http://127.30.208.1:33625/ using document root <none> and password file <none>
I20260807 19:42:55.965804 8743 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:55.965946 8743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:55.971374 8743 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 19:42:55.974176 8781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:55.975587 8743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:55.975752 8743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "a47c7d5502a8424eb5e2288ac10e614a"
format_stamp: "Formatted at 2026-08-07 19:42:53 on dist-test-slave-vq2q"
server_key: "1948c45768e5e6ad0a4547583d649d5c"
server_key_iv: "830c3e489543c9c632b35a6f79c7e5f1"
server_key_version: "encryptionkey@0"
I20260807 19:42:55.976718 8743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:56.006042 8743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:56.010511 8743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:56.010780 8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:56.011884 8743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:56.013597 8743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:56.013700 8743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:56.013829 8743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:56.013928 8743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:56.045217 8743 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:40541
I20260807 19:42:56.045334 8907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:40541 every 8 connection(s)
I20260807 19:42:56.046656 8743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:42:56.048947 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 8743
I20260807 19:42:56.049701 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:35113
--local_ip_for_outbound_sockets=127.30.208.2
--tserver_master_addrs=127.30.208.62:39925
--webserver_port=36599
--webserver_interface=127.30.208.2
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.208.2
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:42:56 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:56 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:42:56.382261 8912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:56.382544 8912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:56.382665 8912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:56.390090 8912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:56.390210 8912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:56.390362 8912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:42:56.400308 8912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.2.keytab
--principal=oryx/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:35113
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=36599
--tserver_master_addrs=127.30.208.62:39925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:56.401700 8912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:56.402980 8912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:56.419000 8921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:56.419039 8918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:56.419076 8919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:56.420744 8912 server_base.cc:1034] running on GCE node
I20260807 19:42:56.421490 8912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:56.422513 8912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:56.423739 8912 hybrid_clock.cc:648] HybridClock initialized: now 1786131776423683 us; error 52 us; skew 500 ppm
I20260807 19:42:56.424057 8912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:56 dist-test-slave-vq2q krb5kdc[8085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131776, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:56.427204 8912 init.cc:377] Logged in from keytab as oryx/127.30.208.2@KRBTEST.COM (short username oryx)
I20260807 19:42:56.428858 8912 webserver.cc:460] Webserver started at http://127.30.208.2:36599/ using document root <none> and password file <none>
I20260807 19:42:56.429627 8912 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:56.429754 8912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:56.435338 8912 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 19:42:56.438009 8928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:56.439337 8912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:42:56.439502 8912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "af8de40424814d89acbd808daa1c9d97"
format_stamp: "Formatted at 2026-08-07 19:42:53 on dist-test-slave-vq2q"
server_key: "e23e19183e6b4a344ee734bd588ae6b9"
server_key_iv: "d7ebdc16ca63d8a366b478313fcc84c2"
server_key_version: "encryptionkey@0"
I20260807 19:42:56.440516 8912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:56.464975 8912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:56.469144 8912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:56.469453 8912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:56.470479 8912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:56.471855 8912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:56.471954 8912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:56.472055 8912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:56.472127 8912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:56.512677 8912 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:35113
I20260807 19:42:56.512732 9054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:35113 every 8 connection(s)
I20260807 19:42:56.514187 8912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:42:56.516563 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 8912
I20260807 19:42:56.517390 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:44443
--local_ip_for_outbound_sockets=127.30.208.3
--tserver_master_addrs=127.30.208.62:39925
--webserver_port=43609
--webserver_interface=127.30.208.3
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.208.3
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:42:56 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:56 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:42:56.871721 9059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:42:56.871994 9059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:42:56.872153 9059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:42:56.879418 9059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:42:56.879539 9059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:42:56.879704 9059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:42:56.889654 9059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/krb5kdc/oryx_127.30.208.3.keytab
--principal=oryx/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:44443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=43609
--tserver_master_addrs=127.30.208.62:39925
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:42:56.891031 9059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:42:56.892309 9059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:42:56.908420 9066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:56.909632 9065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:42:56.917002 9069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:42:56.917037 9059 server_base.cc:1034] running on GCE node
I20260807 19:42:56.918035 9059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:42:56.919086 9059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:42:56.920377 9059 hybrid_clock.cc:648] HybridClock initialized: now 1786131776920322 us; error 49 us; skew 500 ppm
I20260807 19:42:56.920756 9059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:42:56 dist-test-slave-vq2q krb5kdc[8085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131776, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:42:56.924055 9059 init.cc:377] Logged in from keytab as oryx/127.30.208.3@KRBTEST.COM (short username oryx)
I20260807 19:42:56.928925 9059 webserver.cc:460] Webserver started at http://127.30.208.3:43609/ using document root <none> and password file <none>
I20260807 19:42:56.929759 9059 fs_manager.cc:359] Metadata directory not provided
I20260807 19:42:56.929908 9059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:42:56.936044 9059 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 19:42:56.938927 9076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:56.940351 9059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:42:56.940514 9059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "9ad67a2f99fc4544af4c7654e38a3683"
format_stamp: "Formatted at 2026-08-07 19:42:54 on dist-test-slave-vq2q"
server_key: "6075951608454a05827b9239c028676d"
server_key_iv: "c6d44662f09fb1b22d83a650accb60c2"
server_key_version: "encryptionkey@0"
I20260807 19:42:56.941603 9059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:42:56.973917 9059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:42:56.978242 9059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:42:56.978514 9059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:42:56.979602 9059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:42:56.981333 9059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:42:56.981438 9059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:56.981554 9059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:42:56.981662 9059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:42:57.024132 9059 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:44443
I20260807 19:42:57.024173 9203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:44443 every 8 connection(s)
I20260807 19:42:57.025709 9059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:42:57.033321 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 9059
Aug 07 19:42:57 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:57 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:42:57.110666 9210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:57.083997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41786 (local address 127.30.208.62:39925)
0807 19:42:57.089602 (+ 5605us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:57.089607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:57.089624 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:42:57.089680 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:57.089684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:57.089757 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:57.089885 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:57.092897 (+ 3012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.094143 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.096283 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.096946 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.097059 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:57.101068 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:42:57.101080 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:42:57.101103 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:42:57.101154 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:42:57.105043 (+ 3889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:57.105971 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:57.105981 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:57.105991 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:57.106206 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:42:57.106652 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:42:57.106657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:42:57.106661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:42:57.107366 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:42:57.107652 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 19:42:57.109102 (+ 1450us) server_negotiation.cc:299] Negotiation successful
0807 19:42:57.109610 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5466,"thread_start_us":5358,"threads_started":1}
W20260807 19:42:57.356271 8911 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.296s user 0.000s sys 0.013s
I20260807 19:42:57.358907 8911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:56.049829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:40775)
0807 19:42:56.050711 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:56.050753 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:56.052500 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:56.053951 (+ 1451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:56.053963 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:56.054583 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:56.055124 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:56.055144 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:56.056740 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:56.056744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:56.058012 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:56.058026 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:56.059342 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:56.059347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:56.059566 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:56.060190 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:56.060212 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:57.357447 (+1297235us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":448,"threads_started":1}
W20260807 19:42:57.360267 8910 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:56.049618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:40775 (local address 127.30.208.62:39925)
0807 19:42:56.050385 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:56.050401 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:56.050897 (+ 496us) server_negotiation.cc:407] Connection header received
0807 19:42:56.053056 (+ 2159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:56.053075 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:56.053451 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:56.053770 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:56.055320 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:56.056573 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:56.058154 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:56.059108 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:56.059295 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:57.359912 (+1300617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:40775: BlockingRecv error: recv got EOF from 127.30.208.1:40775 (error 108)
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":336,"threads_started":1}
W20260807 19:42:57.360389 8908 heartbeater.cc:644] Failed to heartbeat to 127.30.208.62:39925 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.208.62:39925: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Aug 07 19:42:57 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:57 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
W20260807 19:42:57.375962 8910 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:57.361436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:58751 (local address 127.30.208.62:39925)
0807 19:42:57.362266 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:57.362273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:57.362522 (+ 249us) server_negotiation.cc:407] Connection header received
0807 19:42:57.363306 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:57.363312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:57.363412 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:57.363515 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:57.364784 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.366079 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.368266 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.368973 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.369109 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:57.375652 (+ 6543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:58751: BlockingRecv error: recv got EOF from 127.30.208.1:58751 (error 108)
Metrics: {"server-negotiator.queue_time_us":681}
I20260807 19:42:57.376580 8911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:57.361356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:58751)
0807 19:42:57.362404 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:57.362428 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:57.363127 (+ 699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:57.363725 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:57.363731 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:57.364285 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:57.364541 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:57.364585 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.366303 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.366310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:57.368088 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:57.368110 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.369216 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.369224 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:57.369433 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:57.370256 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:57.370293 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:57.376238 (+ 5945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":896,"thread_start_us":2257,"threads_started":1}
Aug 07 19:42:57 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:57 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:42:57.389941 8911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:57.377497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:54093)
0807 19:42:57.377774 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:57.377794 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:57.377918 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:57.378285 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:57.378291 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:57.378775 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:57.379010 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:57.379022 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.380567 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.380574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:57.382453 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:57.382476 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.383603 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.383609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:57.383815 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:57.384618 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:57.384645 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:57.389579 (+ 4934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146}
W20260807 19:42:57.390184 8910 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:57.377613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:54093 (local address 127.30.208.62:39925)
0807 19:42:57.377793 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:57.377800 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:57.377850 (+ 50us) server_negotiation.cc:407] Connection header received
0807 19:42:57.378037 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:57.378043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:57.378143 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:57.378255 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:57.379159 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.380404 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.382630 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.383420 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.383559 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:57.389863 (+ 6304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:54093: BlockingRecv error: recv got EOF from 127.30.208.1:54093 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 19:42:57.390707 8908 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:42:57.705996 9058 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.165s user 0.005s sys 0.012s
W20260807 19:42:57.707315 9057 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:56.516859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:51895 (local address 127.30.208.62:39925)
0807 19:42:56.529307 (+ 12448us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:56.529313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:56.530139 (+ 826us) server_negotiation.cc:407] Connection header received
0807 19:42:56.532794 (+ 2655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:56.532803 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:56.532915 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:56.533062 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:56.535017 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:56.536333 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:56.538613 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:56.539340 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:56.539484 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:57.706997 (+1167513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:51895: BlockingRecv error: recv got EOF from 127.30.208.2:51895 (error 108)
Metrics: {"server-negotiator.queue_time_us":12279,"thread_start_us":12111,"threads_started":1}
I20260807 19:42:57.707998 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:56.517474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:51895)
0807 19:42:56.529935 (+ 12461us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:56.529992 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:56.532460 (+ 2468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:56.533357 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:56.533370 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:56.534078 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:56.534830 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:56.534855 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:56.536549 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:56.536555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:56.538469 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:56.538486 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:56.539599 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:56.539606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:56.539899 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:56.540737 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:56.540764 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:57.706775 (+1166011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":12131,"thread_start_us":11895,"threads_started":1}
W20260807 19:42:57.708976 9055 heartbeater.cc:644] Failed to heartbeat to 127.30.208.62:39925 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.208.62:39925: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Aug 07 19:42:57 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:57 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:42:57.720656 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:57.709694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:44343)
0807 19:42:57.710034 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:57.710059 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:57.710270 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:57.710572 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:57.710577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:57.711120 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:57.711357 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:57.711369 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.712431 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.712436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:57.714146 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:57.714164 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.714887 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.714894 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:57.715084 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:57.715768 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:57.715804 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:57.720364 (+ 4560us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":78}
W20260807 19:42:57.720804 9057 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:57.710012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:44343 (local address 127.30.208.62:39925)
0807 19:42:57.710175 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:57.710181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:57.710200 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:42:57.710384 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:57.710387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:57.710449 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:57.710525 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:57.711536 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.712268 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.714304 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.714706 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.714812 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:57.720566 (+ 5754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:44343: BlockingRecv error: recv got EOF from 127.30.208.2:44343 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 19:42:57 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:57 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:42:57.731241 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:57.721544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:50217)
0807 19:42:57.721759 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:57.721783 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:57.721914 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:57.722296 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:57.722300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:57.722712 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:57.722908 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:57.722917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.724020 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.724026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:57.725126 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:57.725140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.725898 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.725905 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:57.726096 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:57.726806 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:57.726825 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:57.730860 (+ 4035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 19:42:57.731374 9057 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:57.721781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:50217 (local address 127.30.208.62:39925)
0807 19:42:57.721943 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:57.721949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:57.721967 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:42:57.722036 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:57.722041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:57.722132 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:57.722228 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:57.723060 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.723886 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.725311 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.725743 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.725862 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:57.731112 (+ 5250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:50217: BlockingRecv error: recv got EOF from 127.30.208.2:50217 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 19:42:57.731678 9055 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:42:58.122859 9208 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.027s user 0.000s sys 0.017s
W20260807 19:42:58.124089 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:57.029511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:38487 (local address 127.30.208.62:39925)
0807 19:42:57.081160 (+ 51649us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:57.081167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:57.081222 (+ 55us) server_negotiation.cc:407] Connection header received
0807 19:42:57.085072 (+ 3850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:57.085079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:57.085195 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:57.085286 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:57.089608 (+ 4322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.090641 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.093019 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.093823 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.093950 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:58.123823 (+1029873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:38487: BlockingRecv error: recv got EOF from 127.30.208.3:38487 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":51501,"thread_start_us":54258,"threads_started":1}
I20260807 19:42:58.124694 9208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:57.030045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:38487)
0807 19:42:57.078757 (+ 48712us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:57.078838 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:57.083404 (+ 4566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:57.086106 (+ 2702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:57.086121 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:57.087083 (+ 962us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:57.088046 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:57.088074 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.090802 (+ 2728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.090808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:57.092863 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:57.092878 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:57.094320 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:57.094329 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:57.094634 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:57.095363 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:57.095401 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:58.123607 (+1028206us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48267,"thread_start_us":49579,"threads_started":1}
W20260807 19:42:58.125566 9204 heartbeater.cc:644] Failed to heartbeat to 127.30.208.62:39925 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.208.62:39925: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Aug 07 19:42:58 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:58 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:42:58.139221 9208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:58.126222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:42321)
0807 19:42:58.126411 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:58.126429 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:58.126523 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:58.126782 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:58.126785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:58.127151 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:58.127317 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:58.127327 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.128350 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.128354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:58.129416 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:58.129429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.130092 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.130097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:58.130235 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:58.135530 (+ 5295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:58.135549 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:58.138978 (+ 3429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 19:42:58.139451 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:58.126284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:42321 (local address 127.30.208.62:39925)
0807 19:42:58.126467 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:58.126471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:58.126488 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:42:58.126614 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:58.126618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:58.126679 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:58.126755 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:58.127426 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.128218 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.129543 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.129958 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.130050 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:58.139210 (+ 9160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:42321: BlockingRecv error: recv got EOF from 127.30.208.3:42321 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 19:42:58 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:58 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:42:58.148939 9208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:58.139956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:38969)
0807 19:42:58.140169 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:58.140191 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:58.140329 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:58.140672 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:58.140676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:58.140983 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:58.141144 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:58.141152 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.142269 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.142273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:58.143265 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:58.143279 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.144023 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.144030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:58.144212 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:58.144822 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:58.144846 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:58.148719 (+ 3873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 19:42:58.149274 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:58.140092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:38969 (local address 127.30.208.62:39925)
0807 19:42:58.140381 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:58.140387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:58.140404 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:42:58.140451 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:58.140454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:58.140510 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:58.140587 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:58.141353 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.142117 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.143405 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.143873 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.143972 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:58.148971 (+ 4999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:38969: BlockingRecv error: recv got EOF from 127.30.208.3:38969 (error 108)
Metrics: {"server-negotiator.queue_time_us":156}
W20260807 19:42:58.149333 9204 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 19:42:58 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:58 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:42:58.400933 9213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:58.391317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:35265)
0807 19:42:58.391842 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:58.391863 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:58.392027 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:58.392368 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:58.392373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:58.392832 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:58.392993 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:58.393003 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.394128 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.394134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:58.395421 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:58.395438 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.396177 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.396183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:58.396353 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:58.396983 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:58.397001 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:58.400599 (+ 3598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":286,"threads_started":1}
W20260807 19:42:58.401077 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:58.391586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:35265 (local address 127.30.208.62:39925)
0807 19:42:58.391815 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:58.391822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:58.391970 (+ 148us) server_negotiation.cc:407] Connection header received
0807 19:42:58.392146 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:58.392152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:58.392235 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:58.392318 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:58.393150 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.393979 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.395557 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.396005 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.396120 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:58.400846 (+ 4726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:35265: BlockingRecv error: recv got EOF from 127.30.208.1:35265 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 07 19:42:58 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:58 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:42:58.741125 9214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:58.732253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:37217)
0807 19:42:58.732848 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:58.732864 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:58.732983 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:58.733294 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:58.733298 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:58.733725 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:58.733905 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:58.733914 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.734899 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.734903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:58.735855 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:58.735867 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.736495 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.736499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:58.736612 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:58.737235 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:58.737256 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:58.740885 (+ 3629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":318,"threads_started":1}
W20260807 19:42:58.741362 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:58.732378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:37217 (local address 127.30.208.62:39925)
0807 19:42:58.732556 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:58.732560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:58.732931 (+ 371us) server_negotiation.cc:407] Connection header received
0807 19:42:58.733072 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:58.733076 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:58.733137 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:58.733248 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:58.734024 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.734772 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.735959 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:58.736362 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:58.736449 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:58.741111 (+ 4662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:37217: BlockingRecv error: recv got EOF from 127.30.208.2:37217 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 19:42:59 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:59 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:42:59.158865 9215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:59.149866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:51635)
0807 19:42:59.150436 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:59.150452 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:59.150583 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:59.150909 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:59.150913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:59.151361 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:59.151527 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:59.151537 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.152556 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.152560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:59.153609 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:59.153622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.154394 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.154401 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:59.154558 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:59.155127 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:59.155145 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:59.158524 (+ 3379us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":337,"threads_started":1}
W20260807 19:42:59.158984 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:59.149973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:51635 (local address 127.30.208.62:39925)
0807 19:42:59.150210 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:59.150217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:59.150539 (+ 322us) server_negotiation.cc:407] Connection header received
0807 19:42:59.150694 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:59.150699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:59.150779 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:59.150864 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:59.151676 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.152426 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.153749 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.154215 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.154335 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:59.158767 (+ 4432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:51635: BlockingRecv error: recv got EOF from 127.30.208.3:51635 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 07 19:42:59 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:59 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:42:59.410780 9216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:59.401973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:57983)
0807 19:42:59.402442 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:59.402460 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:59.402610 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:59.402934 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:59.402940 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:59.403385 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:59.403572 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:59.403582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.404582 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.404588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:59.405516 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:59.405529 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.406211 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.406217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:59.406395 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:59.406967 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:59.406985 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:59.410436 (+ 3451us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":245,"threads_started":1}
W20260807 19:42:59.410925 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:59.402098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:57983 (local address 127.30.208.62:39925)
0807 19:42:59.402298 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:59.402305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:59.402546 (+ 241us) server_negotiation.cc:407] Connection header received
0807 19:42:59.402727 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:59.402733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:59.402801 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:59.402879 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:59.403719 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.404450 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.405656 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.406062 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.406171 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:59.410679 (+ 4508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:57983: BlockingRecv error: recv got EOF from 127.30.208.1:57983 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 19:42:59 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:42:59 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:42:59.751118 9217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:42:59.742022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:43005)
0807 19:42:59.742547 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 19:42:59.742563 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:42:59.742699 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:42:59.743020 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:42:59.743025 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:42:59.743540 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:42:59.743726 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:59.743736 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.744795 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.744799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:59.745845 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:42:59.745866 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.746598 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.746604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:42:59.746772 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:59.747319 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:42:59.747336 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:42:59.750869 (+ 3533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":299,"threads_started":1}
W20260807 19:42:59.751247 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:42:59.742109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:43005 (local address 127.30.208.62:39925)
0807 19:42:59.742300 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 19:42:59.742306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:42:59.742648 (+ 342us) server_negotiation.cc:407] Connection header received
0807 19:42:59.742814 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:42:59.742817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:42:59.742888 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:42:59.742968 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:42:59.743895 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.744655 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.745998 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:42:59.746441 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:42:59.746553 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:42:59.751052 (+ 4499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:43005: BlockingRecv error: recv got EOF from 127.30.208.2:43005 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 19:43:00 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:00 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:00.169339 9218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:00.159947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:34359)
0807 19:43:00.160471 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:00.160488 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:00.160610 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:00.161004 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:00.161009 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:00.161558 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:00.161787 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:00.161802 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:00.163169 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:00.163173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:00.164094 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:00.164105 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:00.164707 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:00.164711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:00.164837 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:00.165432 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:00.165450 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:00.169036 (+ 3586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":298,"threads_started":1}
W20260807 19:43:00.169586 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:00.160127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:34359 (local address 127.30.208.62:39925)
0807 19:43:00.160352 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:00.160359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:00.160566 (+ 207us) server_negotiation.cc:407] Connection header received
0807 19:43:00.160717 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:00.160722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:00.160828 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:00.160942 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:00.161962 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:00.163053 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:00.164201 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:00.164593 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:00.164684 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:00.169318 (+ 4634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:34359: BlockingRecv error: recv got EOF from 127.30.208.3:34359 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 19:43:00 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:00 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:00.421207 9219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:00.411895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:45799)
0807 19:43:00.412444 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:00.412461 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:00.412599 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:00.412965 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:00.412969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:00.413433 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:00.413608 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:00.413617 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:00.414641 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:00.414645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:00.415556 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:00.415567 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:00.416276 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:00.416281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:00.416441 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:00.417027 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:00.417049 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:00.420852 (+ 3803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":285,"threads_started":1}
W20260807 19:43:00.421317 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:00.412021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:45799 (local address 127.30.208.62:39925)
0807 19:43:00.412247 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:00.412254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:00.412550 (+ 296us) server_negotiation.cc:407] Connection header received
0807 19:43:00.412713 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:00.412718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:00.412822 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:00.412906 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:00.413767 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:00.414513 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:00.415693 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:00.416128 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:00.416236 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:00.421062 (+ 4826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:45799: BlockingRecv error: recv got EOF from 127.30.208.1:45799 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 07 19:43:00 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:00 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:00.761462 9220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:00.752166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:34903)
0807 19:43:00.752777 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:00.752795 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:00.752931 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:00.753234 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:00.753237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:00.753614 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:00.753791 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:00.753800 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:00.755240 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:00.755245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:00.756282 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:00.756299 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:00.756999 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:00.757005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:00.757202 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:00.757774 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:00.757792 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:00.761125 (+ 3333us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":342,"threads_started":1}
W20260807 19:43:00.761677 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:00.752279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:34903 (local address 127.30.208.62:39925)
0807 19:43:00.752463 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:00.752467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:00.752894 (+ 427us) server_negotiation.cc:407] Connection header received
0807 19:43:00.753029 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:00.753033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:00.753098 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:00.753213 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:00.753984 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:00.755092 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:00.756442 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:00.756851 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:00.756967 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:00.761381 (+ 4414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:34903: BlockingRecv error: recv got EOF from 127.30.208.2:34903 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 19:43:01 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:01 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:01.179450 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:01.170256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:47143)
0807 19:43:01.170747 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:01.170763 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:01.170892 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:01.171247 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:01.171251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:01.171677 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:01.171865 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:01.171875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:01.173166 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:01.173206 (+ 40us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:01.174140 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:01.174154 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:01.174858 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:01.174863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:01.175002 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:01.175608 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:01.175627 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:01.179140 (+ 3513us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":246,"threads_started":1}
W20260807 19:43:01.179658 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:01.170339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:47143 (local address 127.30.208.62:39925)
0807 19:43:01.170527 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:01.170531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:01.170828 (+ 297us) server_negotiation.cc:407] Connection header received
0807 19:43:01.170982 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:01.170985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:01.171058 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:01.171143 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:01.172036 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:01.172965 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:01.174305 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:01.174692 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:01.174799 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:01.179411 (+ 4612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:47143: BlockingRecv error: recv got EOF from 127.30.208.3:47143 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 19:43:01 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:01 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:01.431542 9222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:01.422379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:35915)
0807 19:43:01.422951 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:01.422969 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:01.423119 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:01.423711 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:01.423715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:01.424134 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:01.424332 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:01.424342 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:01.425453 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:01.425458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:01.426495 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:01.426509 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:01.427164 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:01.427168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:01.427286 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:01.427901 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:01.427919 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:01.431250 (+ 3331us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":280,"threads_started":1}
W20260807 19:43:01.431674 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:01.422496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:35915 (local address 127.30.208.62:39925)
0807 19:43:01.422739 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:01.422745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:01.423088 (+ 343us) server_negotiation.cc:407] Connection header received
0807 19:43:01.423259 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:01.423263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:01.423352 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:01.423467 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:01.424475 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:01.425295 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:01.426634 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:01.427054 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:01.427148 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:01.431461 (+ 4313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:35915: BlockingRecv error: recv got EOF from 127.30.208.1:35915 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 19:43:01 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:01 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:01.771522 9223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:01.762558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:36565)
0807 19:43:01.763123 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:01.763140 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:01.763269 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:01.763609 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:01.763614 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:01.764013 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:01.764174 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:01.764184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:01.765220 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:01.765224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:01.766230 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:01.766242 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:01.766965 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:01.766971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:01.767136 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:01.767773 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:01.767791 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:01.771195 (+ 3404us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":300,"threads_started":1}
W20260807 19:43:01.771634 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:01.762653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:36565 (local address 127.30.208.62:39925)
0807 19:43:01.762887 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:01.762894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:01.763224 (+ 330us) server_negotiation.cc:407] Connection header received
0807 19:43:01.763400 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:01.763404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:01.763467 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:01.763546 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:01.764317 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:01.765055 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:01.766379 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:01.766814 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:01.766935 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:01.771386 (+ 4451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:36565: BlockingRecv error: recv got EOF from 127.30.208.2:36565 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 19:43:02 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:02 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:02.189481 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:02.180536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:49445)
0807 19:43:02.181075 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:02.181096 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:02.181293 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:02.181656 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:02.181661 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:02.182142 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:02.182319 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:02.182329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:02.183348 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:02.183352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:02.184329 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:02.184341 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:02.185071 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:02.185076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:02.185284 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:02.185830 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:02.185848 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:02.189205 (+ 3357us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":433,"threads_started":1}
W20260807 19:43:02.189774 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:02.180647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:49445 (local address 127.30.208.62:39925)
0807 19:43:02.180818 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:02.180822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:02.181171 (+ 349us) server_negotiation.cc:407] Connection header received
0807 19:43:02.181426 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:02.181432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:02.181519 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:02.181600 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:02.182465 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:02.183198 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:02.184472 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:02.184910 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:02.185026 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:02.189497 (+ 4471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:49445: BlockingRecv error: recv got EOF from 127.30.208.3:49445 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 19:43:02 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:02 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:02.442082 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:02.432735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:51337)
0807 19:43:02.433305 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:02.433321 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:02.433460 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:02.433836 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:02.433841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:02.434421 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:02.434606 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:02.434619 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:02.435888 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:02.435894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:02.436863 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:02.436876 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:02.437609 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:02.437614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:02.437754 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:02.438360 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:02.438379 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:02.441848 (+ 3469us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":287,"threads_started":1}
W20260807 19:43:02.442353 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:02.432917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:51337 (local address 127.30.208.62:39925)
0807 19:43:02.433143 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:02.433150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:02.433404 (+ 254us) server_negotiation.cc:407] Connection header received
0807 19:43:02.433592 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:02.433596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:02.433656 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:02.433786 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:02.434823 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:02.435741 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:02.437011 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:02.437459 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:02.437559 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:02.442086 (+ 4527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:51337: BlockingRecv error: recv got EOF from 127.30.208.1:51337 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 07 19:43:02 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:02 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:02.781385 9226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:02.772491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:34273)
0807 19:43:02.772996 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:02.773012 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:02.773136 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:02.773477 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:02.773481 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:02.773983 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:02.774158 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:02.774168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:02.775120 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:02.775124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:02.776073 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:02.776085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:02.776858 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:02.776862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:02.776991 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:02.777613 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:02.777639 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:02.781081 (+ 3442us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":258,"threads_started":1}
W20260807 19:43:02.781562 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:02.772624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:34273 (local address 127.30.208.62:39925)
0807 19:43:02.772776 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:02.772782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:02.773097 (+ 315us) server_negotiation.cc:407] Connection header received
0807 19:43:02.773284 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:02.773288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:02.773351 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:02.773430 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:02.774280 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:02.774996 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:02.776214 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:02.776691 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:02.776786 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:02.781344 (+ 4558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:34273: BlockingRecv error: recv got EOF from 127.30.208.2:34273 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 07 19:43:03 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:03 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:03.204116 9227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:03.190556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:59173)
0807 19:43:03.191082 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:03.191099 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:03.191239 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:03.191582 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:03.191587 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:03.192041 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:03.192189 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:03.192199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:03.193237 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:03.193240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:03.194174 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:03.194185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:03.194870 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:03.194876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:03.195035 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:03.200331 (+ 5296us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:03.200352 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:03.203870 (+ 3518us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":265,"threads_started":1}
W20260807 19:43:03.204360 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:03.190709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:59173 (local address 127.30.208.62:39925)
0807 19:43:03.190941 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:03.190948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:03.191184 (+ 236us) server_negotiation.cc:407] Connection header received
0807 19:43:03.191364 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:03.191369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:03.191446 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:03.191529 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:03.192335 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:03.193067 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:03.194314 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:03.194732 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:03.194842 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:03.204102 (+ 9260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:59173: BlockingRecv error: recv got EOF from 127.30.208.3:59173 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 07 19:43:03 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:03 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:03.452418 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:03.442976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:44853)
0807 19:43:03.443517 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:03.443533 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:03.443672 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:03.443989 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:03.443994 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:03.444431 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:03.444627 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:03.444639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:03.445806 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:03.445812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:03.446812 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:03.446825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:03.447559 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:03.447564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:03.447728 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:03.448285 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:03.448304 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:03.452153 (+ 3849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":286,"threads_started":1}
W20260807 19:43:03.452564 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:03.443062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:44853 (local address 127.30.208.62:39925)
0807 19:43:03.443307 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:03.443313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:03.443614 (+ 301us) server_negotiation.cc:407] Connection header received
0807 19:43:03.443785 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:03.443790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:03.443856 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:03.443938 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:03.444763 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:03.445638 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:03.446954 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:03.447398 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:03.447516 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:03.452358 (+ 4842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:44853: BlockingRecv error: recv got EOF from 127.30.208.1:44853 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 19:43:03 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:03 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:03.791613 9229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:03.782578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:36917)
0807 19:43:03.783221 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:03.783242 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:03.783392 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:03.783754 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:03.783758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:03.784225 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:03.784438 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:03.784451 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:03.785519 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:03.785525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:03.786488 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:03.786501 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:03.787212 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:03.787218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:03.787404 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:03.787956 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:03.787973 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:03.791281 (+ 3308us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":404,"threads_started":1}
W20260807 19:43:03.791736 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:03.782800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:36917 (local address 127.30.208.62:39925)
0807 19:43:03.783006 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:03.783011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:03.783324 (+ 313us) server_negotiation.cc:407] Connection header received
0807 19:43:03.783524 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:03.783529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:03.783611 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:03.783707 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:03.784565 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:03.785369 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:03.786625 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:03.787059 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:03.787172 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:03.791532 (+ 4360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:36917: BlockingRecv error: recv got EOF from 127.30.208.2:36917 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 19:43:04 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:04 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:04.214105 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:04.205161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:56905)
0807 19:43:04.205647 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:04.205677 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:04.205798 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:04.206097 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:04.206100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:04.206530 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:04.206702 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:04.206711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:04.207720 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:04.207724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:04.208673 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:04.208686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:04.209419 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:04.209425 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:04.209567 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:04.210125 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:04.210144 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:04.213862 (+ 3718us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":249,"threads_started":1}
W20260807 19:43:04.214325 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:04.205300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:56905 (local address 127.30.208.62:39925)
0807 19:43:04.205519 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:04.205526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:04.205755 (+ 229us) server_negotiation.cc:407] Connection header received
0807 19:43:04.205901 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:04.205905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:04.205983 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:04.206063 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:04.206822 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:04.207588 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:04.208813 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:04.209258 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:04.209351 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:04.214093 (+ 4742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:56905: BlockingRecv error: recv got EOF from 127.30.208.3:56905 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 19:43:04 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:04 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:04.462745 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:04.453398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:43201)
0807 19:43:04.454033 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:04.454050 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:04.454178 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:04.454464 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:04.454467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:04.454960 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:04.455189 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:04.455201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:04.456531 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:04.456535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:04.457517 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:04.457531 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:04.458253 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:04.458258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:04.458415 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:04.458986 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:04.459004 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:04.462404 (+ 3400us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":303,"threads_started":1}
W20260807 19:43:04.462862 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:04.453520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:43201 (local address 127.30.208.62:39925)
0807 19:43:04.453688 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:04.453693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:04.454121 (+ 428us) server_negotiation.cc:407] Connection header received
0807 19:43:04.454282 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:04.454285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:04.454348 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:04.454430 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:04.455332 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:04.456383 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:04.457661 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:04.458117 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:04.458210 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:04.462622 (+ 4412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:43201: BlockingRecv error: recv got EOF from 127.30.208.1:43201 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 19:43:04 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:04 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:04.801620 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:04.792716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:55833)
0807 19:43:04.793369 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:04.793386 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:04.793518 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:04.793842 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:04.793846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:04.794263 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:04.794437 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:04.794447 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:04.795482 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:04.795487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:04.796441 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:04.796455 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:04.797201 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:04.797208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:04.797371 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:04.797908 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:04.797926 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:04.801363 (+ 3437us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":286,"threads_started":1}
W20260807 19:43:04.801829 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:04.792773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:55833 (local address 127.30.208.62:39925)
0807 19:43:04.793023 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:04.793029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:04.793474 (+ 445us) server_negotiation.cc:407] Connection header received
0807 19:43:04.793631 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:04.793637 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:04.793705 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:04.793783 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:04.794582 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:04.795338 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:04.796587 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:04.797024 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:04.797142 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:04.801589 (+ 4447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:55833: BlockingRecv error: recv got EOF from 127.30.208.2:55833 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Aug 07 19:43:05 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:05 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:05.224764 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:05.215201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:48247)
0807 19:43:05.215829 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:05.215846 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:05.215971 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:05.216396 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:05.216401 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:05.216830 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:05.217056 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:05.217068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:05.218392 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:05.218396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:05.219385 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:05.219398 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:05.220171 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:05.220176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:05.220308 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:05.220925 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:05.220944 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:05.224498 (+ 3554us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":292,"threads_started":1}
W20260807 19:43:05.224936 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:05.215327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:48247 (local address 127.30.208.62:39925)
0807 19:43:05.215503 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:05.215508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:05.215961 (+ 453us) server_negotiation.cc:407] Connection header received
0807 19:43:05.216065 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:05.216071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:05.216179 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:05.216301 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:05.217224 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:05.218201 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:05.219521 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:05.219933 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:05.220025 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:05.224723 (+ 4698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:48247: BlockingRecv error: recv got EOF from 127.30.208.3:48247 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 19:43:05 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:05 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:05.472640 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:05.463687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:42635)
0807 19:43:05.464200 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:05.464218 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:05.464330 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:05.464653 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:05.464657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:05.465108 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:05.465323 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:05.465334 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:05.466401 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:05.466405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:05.467353 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:05.467365 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:05.468146 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:05.468152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:05.468287 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:05.468894 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:05.468913 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:05.472298 (+ 3385us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":269,"threads_started":1}
W20260807 19:43:05.472755 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:05.463861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:42635 (local address 127.30.208.62:39925)
0807 19:43:05.464100 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:05.464106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:05.464288 (+ 182us) server_negotiation.cc:407] Connection header received
0807 19:43:05.464420 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:05.464424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:05.464487 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:05.464566 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:05.465467 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:05.466238 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:05.467507 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:05.467964 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:05.468063 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:05.472533 (+ 4470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:42635: BlockingRecv error: recv got EOF from 127.30.208.1:42635 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 07 19:43:05 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:05 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:05.813637 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:05.802721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:58871)
0807 19:43:05.803292 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:05.803317 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:05.803445 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:05.803956 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:05.803959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:05.804673 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:05.804903 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:05.804917 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:05.806451 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:05.806456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:05.807429 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:05.807443 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:05.808417 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:05.808426 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:05.808630 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:05.809539 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:05.809564 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:05.813378 (+ 3814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":299,"threads_started":1}
W20260807 19:43:05.814020 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:05.802899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:58871 (local address 127.30.208.62:39925)
0807 19:43:05.803062 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:05.803067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:05.803402 (+ 335us) server_negotiation.cc:407] Connection header received
0807 19:43:05.803622 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:05.803627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:05.803725 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:05.803896 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:05.805094 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:05.806295 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:05.807581 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:05.808057 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:05.808159 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:05.813683 (+ 5524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:58871: BlockingRecv error: recv got EOF from 127.30.208.2:58871 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 19:43:06 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:06 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:06.235095 9237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:06.225766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:36265)
0807 19:43:06.226301 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:06.226319 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:06.226436 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:06.226797 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:06.226801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:06.227240 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:06.227418 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:06.227428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.228547 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.228557 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:06.229582 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:06.229597 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.230369 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.230375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:06.230493 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:06.231069 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:06.231087 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:06.234764 (+ 3677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":277,"threads_started":1}
W20260807 19:43:06.235265 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:06.225871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:36265 (local address 127.30.208.62:39925)
0807 19:43:06.226026 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:06.226031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:06.226385 (+ 354us) server_negotiation.cc:407] Connection header received
0807 19:43:06.226545 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:06.226549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:06.226613 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:06.226686 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:06.227567 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.228399 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.229735 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.230183 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.230290 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:06.235035 (+ 4745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:36265: BlockingRecv error: recv got EOF from 127.30.208.3:36265 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 19:43:06 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:06 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:06.482208 9238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:06.473712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:36603)
0807 19:43:06.474245 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:06.474266 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:06.474406 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:06.474682 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:06.474685 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:06.475083 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:06.475226 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:06.475235 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.476279 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.476282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:06.477244 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:06.477260 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.477953 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.477957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:06.478080 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:06.478613 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:06.478630 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:06.481974 (+ 3344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":283,"threads_started":1}
W20260807 19:43:06.482373 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:06.473783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:36603 (local address 127.30.208.62:39925)
0807 19:43:06.474004 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:06.474011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:06.474364 (+ 353us) server_negotiation.cc:407] Connection header received
0807 19:43:06.474499 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:06.474503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:06.474563 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:06.474638 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:06.475360 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.476174 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.477379 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.477832 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.477922 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:06.482164 (+ 4242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:36603: BlockingRecv error: recv got EOF from 127.30.208.1:36603 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 19:43:06 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:06 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:06.823457 9240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:06.814679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:33991)
0807 19:43:06.815145 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:06.815161 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:06.815271 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:06.815558 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:06.815561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:06.815979 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:06.816127 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:06.816135 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.817151 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.817155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:06.818169 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:06.818181 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.818841 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.818845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:06.818983 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:06.819572 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:06.819590 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:06.823217 (+ 3627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":238,"threads_started":1}
W20260807 19:43:06.823601 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:06.814745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:33991 (local address 127.30.208.62:39925)
0807 19:43:06.814952 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:06.814958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:06.815234 (+ 276us) server_negotiation.cc:407] Connection header received
0807 19:43:06.815372 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:06.815376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:06.815439 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:06.815516 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:06.816268 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.817016 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.818275 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:06.818685 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:06.818772 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:06.823380 (+ 4608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:33991: BlockingRecv error: recv got EOF from 127.30.208.2:33991 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 19:43:07 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:07 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:07.246084 9241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:07.236305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:58285)
0807 19:43:07.236865 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:07.236882 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:07.237008 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:07.237490 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:07.237495 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:07.237949 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:07.238106 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:07.238116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:07.239152 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:07.239156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:07.240186 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:07.240200 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:07.240949 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:07.240954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:07.241083 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:07.241723 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:07.241741 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:07.245758 (+ 4017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":269,"threads_started":1}
W20260807 19:43:07.246132 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:07.236420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:58285 (local address 127.30.208.62:39925)
0807 19:43:07.236609 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:07.236614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:07.236964 (+ 350us) server_negotiation.cc:407] Connection header received
0807 19:43:07.237111 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:07.237115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:07.237223 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:07.237321 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:07.238267 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:07.239026 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:07.240334 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:07.240801 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:07.240892 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:07.245924 (+ 5032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:58285: BlockingRecv error: recv got EOF from 127.30.208.3:58285 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 19:43:07 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:07 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:07.491696 9242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:07.483087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:60323)
0807 19:43:07.483587 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:07.483604 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:07.483722 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:07.483994 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:07.483997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:07.484365 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:07.484530 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:07.484539 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:07.485568 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:07.485572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:07.486601 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:07.486616 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:07.487354 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:07.487359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:07.487477 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:07.488110 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:07.488129 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:07.491456 (+ 3327us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":254,"threads_started":1}
W20260807 19:43:07.491941 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:07.483175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:60323 (local address 127.30.208.62:39925)
0807 19:43:07.483356 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:07.483361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:07.483668 (+ 307us) server_negotiation.cc:407] Connection header received
0807 19:43:07.483818 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:07.483822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:07.483883 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:07.483952 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:07.484640 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:07.485414 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:07.486742 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:07.487191 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:07.487292 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:07.491674 (+ 4382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:60323: BlockingRecv error: recv got EOF from 127.30.208.1:60323 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 19:43:07 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:07 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:07.833439 9243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:07.824397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:43355)
0807 19:43:07.824892 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:07.824907 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:07.825019 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:07.825317 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:07.825320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:07.825742 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:07.825906 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:07.825914 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:07.826988 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:07.826992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:07.827979 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:07.827991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:07.828719 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:07.828726 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:07.828844 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:07.829511 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:07.829530 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:07.833145 (+ 3615us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":264,"threads_started":1}
W20260807 19:43:07.833595 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:07.824494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:43355 (local address 127.30.208.62:39925)
0807 19:43:07.824703 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:07.824708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:07.824980 (+ 272us) server_negotiation.cc:407] Connection header received
0807 19:43:07.825108 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:07.825111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:07.825200 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:07.825282 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:07.826076 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:07.826825 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:07.828145 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:07.828559 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:07.828714 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:07.833389 (+ 4675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:43355: BlockingRecv error: recv got EOF from 127.30.208.2:43355 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 19:43:08 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:08 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:08.259447 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:08.247093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:52933)
0807 19:43:08.247598 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:08.247618 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:08.247755 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:08.248068 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:08.248073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:08.248490 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:08.248661 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:08.248670 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.249757 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.249762 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:08.250734 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:08.250746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.251464 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.251470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:08.251626 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:08.255478 (+ 3852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:08.255501 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:08.259106 (+ 3605us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":241,"threads_started":1}
W20260807 19:43:08.259609 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:08.247360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:52933 (local address 127.30.208.62:39925)
0807 19:43:08.247593 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:08.247600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:08.247691 (+ 91us) server_negotiation.cc:407] Connection header received
0807 19:43:08.247870 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:08.247876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:08.247941 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:08.248022 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:08.248811 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.249613 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.250876 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.251310 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.251427 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:08.259345 (+ 7918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:52933: BlockingRecv error: recv got EOF from 127.30.208.3:52933 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 07 19:43:08 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:08 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:08.501314 9245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:08.492695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:46369)
0807 19:43:08.493167 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:08.493210 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:08.493354 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:08.493671 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:08.493676 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:08.494127 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:08.494302 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:08.494312 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.495315 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.495321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:08.496218 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:08.496230 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.496928 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.496933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:08.497093 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:08.497647 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:08.497665 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:08.500979 (+ 3314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":357,"threads_started":1}
W20260807 19:43:08.501447 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:08.492772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:46369 (local address 127.30.208.62:39925)
0807 19:43:08.493053 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:08.493059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:08.493303 (+ 244us) server_negotiation.cc:407] Connection header received
0807 19:43:08.493468 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:08.493473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:08.493541 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:08.493627 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:08.494444 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.495166 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.496357 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.496782 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.496890 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:08.501206 (+ 4316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:46369: BlockingRecv error: recv got EOF from 127.30.208.1:46369 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 07 19:43:08 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:08 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:08.843098 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:08.834341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:58349)
0807 19:43:08.834779 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:08.834795 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:08.834916 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:08.835172 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:08.835175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:08.835610 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:08.835783 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:08.835792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.836783 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.836787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:08.837826 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:08.837839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.838487 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.838491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:08.838609 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:08.839159 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:08.839177 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:08.842875 (+ 3698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":236,"threads_started":1}
W20260807 19:43:08.843238 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:08.834455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:58349 (local address 127.30.208.62:39925)
0807 19:43:08.834669 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:08.834674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:08.834872 (+ 198us) server_negotiation.cc:407] Connection header received
0807 19:43:08.835005 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:08.835009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:08.835071 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:08.835146 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:08.835896 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.836673 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.837946 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:08.838367 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:08.838463 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:08.843038 (+ 4575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:58349: BlockingRecv error: recv got EOF from 127.30.208.2:58349 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 07 19:43:09 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:09 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:09.270300 9247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:09.260571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:41571)
0807 19:43:09.261110 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:09.261127 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:09.261294 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:09.261616 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:09.261620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:09.262105 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:09.262314 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:09.262328 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:09.263535 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:09.263539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:09.264881 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:09.264894 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:09.265922 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:09.265929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:09.266117 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:09.266790 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:09.266821 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:09.270073 (+ 3252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":314,"threads_started":1}
W20260807 19:43:09.270524 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:09.260727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:41571 (local address 127.30.208.62:39925)
0807 19:43:09.260955 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:09.260961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:09.261207 (+ 246us) server_negotiation.cc:407] Connection header received
0807 19:43:09.261414 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:09.261420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:09.261489 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:09.261572 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:09.262486 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:09.263411 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:09.265029 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:09.265705 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:09.265842 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:09.270286 (+ 4444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:41571: BlockingRecv error: recv got EOF from 127.30.208.3:41571 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 07 19:43:09 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:09 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:09.511639 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:09.502436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:34845)
0807 19:43:09.503006 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:09.503024 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:09.503144 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:09.503438 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:09.503441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:09.503929 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:09.504143 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:09.504157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:09.505476 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:09.505482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:09.506457 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:09.506470 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:09.507204 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:09.507209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:09.507344 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:09.507925 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:09.507944 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:09.511294 (+ 3350us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":329,"threads_started":1}
W20260807 19:43:09.511754 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:09.502513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:34845 (local address 127.30.208.62:39925)
0807 19:43:09.502742 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:09.502746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:09.503096 (+ 350us) server_negotiation.cc:407] Connection header received
0807 19:43:09.503258 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:09.503262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:09.503331 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:09.503406 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:09.504335 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:09.505337 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:09.506580 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:09.507038 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:09.507127 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:09.511507 (+ 4380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:34845: BlockingRecv error: recv got EOF from 127.30.208.1:34845 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 19:43:09 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:09 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:09.853145 9249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:09.843999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:49625)
0807 19:43:09.844576 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:09.844594 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:09.844745 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:09.845041 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:09.845045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:09.845513 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:09.845723 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:09.845734 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:09.846767 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:09.846773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:09.847828 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:09.847840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:09.848579 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:09.848586 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:09.848746 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:09.849484 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:09.849505 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:09.852807 (+ 3302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":304,"threads_started":1}
W20260807 19:43:09.853291 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:09.844107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:49625 (local address 127.30.208.62:39925)
0807 19:43:09.844348 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:09.844355 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:09.844695 (+ 340us) server_negotiation.cc:407] Connection header received
0807 19:43:09.844850 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:09.844853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:09.844921 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:09.845027 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:09.845870 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:09.846622 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:09.847956 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:09.848402 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:09.848511 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:09.853004 (+ 4493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:49625: BlockingRecv error: recv got EOF from 127.30.208.2:49625 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 19:43:10 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:10 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:10.280192 9250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:10.271212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:38037)
0807 19:43:10.271763 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:10.271779 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:10.271914 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:10.272185 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:10.272188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:10.272607 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:10.272779 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:10.272788 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:10.273954 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:10.273960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:10.275008 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:10.275022 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:10.275732 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:10.275739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:10.275925 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:10.276497 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:10.276514 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:10.279893 (+ 3379us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":275,"threads_started":1}
W20260807 19:43:10.280349 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:10.271268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:38037 (local address 127.30.208.62:39925)
0807 19:43:10.271422 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:10.271427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:10.271849 (+ 422us) server_negotiation.cc:407] Connection header received
0807 19:43:10.272006 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:10.272010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:10.272072 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:10.272153 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:10.272908 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:10.273808 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:10.275143 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:10.275552 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:10.275675 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:10.280099 (+ 4424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:38037: BlockingRecv error: recv got EOF from 127.30.208.3:38037 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 19:43:10 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:10 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:10.522068 9251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:10.512779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:52541)
0807 19:43:10.513276 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:10.513293 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:10.513413 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:10.513668 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:10.513671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:10.514258 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:10.514455 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:10.514468 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:10.515473 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:10.515479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:10.516893 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:10.516909 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:10.517630 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:10.517635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:10.517763 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:10.518311 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:10.518333 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:10.521828 (+ 3495us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":247,"threads_started":1}
W20260807 19:43:10.522222 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:10.512906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:52541 (local address 127.30.208.62:39925)
0807 19:43:10.513103 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:10.513107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:10.513358 (+ 251us) server_negotiation.cc:407] Connection header received
0807 19:43:10.513495 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:10.513499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:10.513560 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:10.513640 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:10.514596 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:10.515331 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:10.517031 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:10.517507 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:10.517603 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:10.522025 (+ 4422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:52541: BlockingRecv error: recv got EOF from 127.30.208.1:52541 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 19:43:10 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:10 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:10.867110 9253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:10.854286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:54723)
0807 19:43:10.854835 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:10.854851 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:10.854984 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:10.855339 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:10.855342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:10.855799 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:10.855950 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:10.855960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:10.856952 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:10.856956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:10.857933 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:10.857946 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:10.858633 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:10.858638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:10.858804 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:10.863269 (+ 4465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:10.863289 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:10.866851 (+ 3562us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":280,"threads_started":1}
W20260807 19:43:10.867295 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:10.854472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:54723 (local address 127.30.208.62:39925)
0807 19:43:10.854698 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:10.854704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:10.854950 (+ 246us) server_negotiation.cc:407] Connection header received
0807 19:43:10.855125 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:10.855130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:10.855200 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:10.855290 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:10.856067 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:10.856812 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:10.858052 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:10.858501 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:10.858594 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:10.867082 (+ 8488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:54723: BlockingRecv error: recv got EOF from 127.30.208.2:54723 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 19:43:11 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:11 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.3@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:11.290431 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:11.281328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.3:49503)
0807 19:43:11.281811 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:11.281831 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:11.281950 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:11.282227 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:11.282230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:11.282662 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:11.282815 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:11.282826 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:11.283838 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:11.283842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:11.284770 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:11.284783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:11.285683 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:11.285690 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:11.285872 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:11.286635 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:11.286660 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:11.290077 (+ 3417us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":260,"threads_started":1}
W20260807 19:43:11.290555 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:11.281457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:49503 (local address 127.30.208.62:39925)
0807 19:43:11.281622 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:11.281626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:11.281899 (+ 273us) server_negotiation.cc:407] Connection header received
0807 19:43:11.282045 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:11.282049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:11.282110 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:11.282204 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:11.282946 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:11.283709 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:11.284941 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:11.285516 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:11.285654 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:11.290332 (+ 4678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.3:49503: BlockingRecv error: recv got EOF from 127.30.208.3:49503 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 19:43:11 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:11 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.1@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:11.532984 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:11.523254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.1:34881)
0807 19:43:11.523845 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:11.523862 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:11.523995 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:11.524318 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:11.524323 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:11.524754 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:11.524942 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:11.524952 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:11.526049 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:11.526053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:11.526985 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:11.527000 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:11.527893 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:11.527899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:11.528069 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:11.528754 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:11.528778 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:11.532621 (+ 3843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":261,"threads_started":1}
W20260807 19:43:11.533069 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:11.523391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:34881 (local address 127.30.208.62:39925)
0807 19:43:11.523730 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:11.523736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:11.523942 (+ 206us) server_negotiation.cc:407] Connection header received
0807 19:43:11.524107 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:11.524112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:11.524180 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:11.524262 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:11.525083 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:11.525891 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:11.527163 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:11.527744 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:11.527857 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:11.532826 (+ 4969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.1:34881: BlockingRecv error: recv got EOF from 127.30.208.1:34881 (error 108)
Metrics: {"server-negotiator.queue_time_us":180}
Aug 07 19:43:11 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 07 19:43:11 dist-test-slave-vq2q krb5kdc[8085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.208.2@KRBTEST.COM for oryx/127.30.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260807 19:43:11.876997 9257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:11.868330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:39925 (local address 127.30.208.2:51713)
0807 19:43:11.868891 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:11.868907 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:11.869016 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:11.869373 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:11.869378 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:11.869753 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:11.869948 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:11.869957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:11.870957 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:11.870960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:11.871872 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:11.871884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:11.872578 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:11.872583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:11.872747 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:11.873332 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:11.873351 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:11.876625 (+ 3274us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.208.62:39925: Server oryx/127.30.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":308,"threads_started":1}
W20260807 19:43:11.877092 9207 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:11.868416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:51713 (local address 127.30.208.62:39925)
0807 19:43:11.868610 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:11.868617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:11.868998 (+ 381us) server_negotiation.cc:407] Connection header received
0807 19:43:11.869121 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:11.869125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:11.869210 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:11.869315 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:11.870093 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:11.870817 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:11.872013 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:11.872441 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:11.872550 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:11.876870 (+ 4320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.208.2:51713: BlockingRecv error: recv got EOF from 127.30.208.2:51713 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:43:12.038194 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 8743
I20260807 19:43:12.051317 9260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:12.039431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36328 (local address 127.30.208.1:40541)
0807 19:43:12.040066 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:12.040074 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:12.040101 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:43:12.040165 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:12.040171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:12.040447 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:12.040610 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:12.042015 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:12.043098 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:12.044337 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:12.044928 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:12.045048 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:12.048033 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:12.048041 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:12.048054 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:12.048099 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:12.050823 (+ 2724us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.208.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 19:43:12.051112 (+ 289us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36328: Request ticket server kudu/127.30.208.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":325,"threads_started":1}
W20260807 19:43:12.051429 9260 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36328: Request ticket server kudu/127.30.208.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 19:43:12.051525 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 8743: Not authorized: Client connection negotiation failed: client connection to 127.30.208.1:40541: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.208.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 19:43:12.051712 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 8743
I20260807 19:43:12.069756 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 8912
I20260807 19:43:12.082221 9261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:12.070899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.30.208.2:35113)
0807 19:43:12.071594 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:12.071603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:12.071635 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:43:12.071713 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:12.071721 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:12.072023 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:12.072245 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:12.073577 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:12.074722 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:12.075926 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:12.076634 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:12.076737 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:12.079191 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:12.079200 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:12.079215 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:12.079257 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:12.081690 (+ 2433us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.208.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 19:43:12.081967 (+ 277us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44676: Request ticket server kudu/127.30.208.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":305,"threads_started":1}
W20260807 19:43:12.082326 9261 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44676: Request ticket server kudu/127.30.208.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 19:43:12.082365 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 8912: Not authorized: Client connection negotiation failed: client connection to 127.30.208.2:35113: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.208.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 19:43:12.082540 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 8912
I20260807 19:43:12.100996 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 9059
I20260807 19:43:12.114039 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:12.102211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45690 (local address 127.30.208.3:44443)
0807 19:43:12.102873 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:12.102881 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:12.102910 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:43:12.102983 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:12.102989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:12.103237 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:12.103446 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:12.104916 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:12.106041 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:12.107549 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:12.108100 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:12.108211 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:12.110771 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:12.110780 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:12.110794 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:12.110838 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:12.113507 (+ 2669us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.208.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 19:43:12.113795 (+ 288us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45690: Request ticket server kudu/127.30.208.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":353,"threads_started":1}
W20260807 19:43:12.114149 9262 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45690: Request ticket server kudu/127.30.208.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 19:43:12.114280 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 9059: Not authorized: Client connection negotiation failed: client connection to 127.30.208.3:44443: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.30.208.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 19:43:12.114459 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 9059
I20260807 19:43:12.132886 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 8667
I20260807 19:43:12.134150 8735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:12.255228 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 8667
2026-08-07T19:43:12Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21205 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T19:43:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:12Z Disabled control of system clock
I20260807 19:43:12.337268 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:44521
--webserver_interface=127.30.208.62
--webserver_port=0
--builtin_ntp_servers=127.30.208.20:36959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:44521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 19:43:12.671792 9267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:12.672083 9267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:12.672174 9267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:12.679435 9267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:12.679550 9267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:12.679589 9267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:12.679644 9267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:12.679692 9267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:12.689023 9267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:44521
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:44521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:43:12.690346 9267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:12.692101 9267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:12.703969 9273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:12.703915 9272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:12.703974 9276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:12.704676 9267 server_base.cc:1034] running on GCE node
I20260807 19:43:12.705500 9267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:12.706949 9267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:12.708158 9267 hybrid_clock.cc:648] HybridClock initialized: now 1786131792708121 us; error 26 us; skew 500 ppm
I20260807 19:43:12.708498 9267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:43:12.710840 9267 webserver.cc:460] Webserver started at http://127.30.208.62:40339/ using document root <none> and password file <none>
I20260807 19:43:12.711547 9267 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:12.711654 9267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:12.711988 9267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:12.714905 9267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "55b7387ad8f64bafab088fb1c76a6fc6"
format_stamp: "Formatted at 2026-08-07 19:43:12 on dist-test-slave-vq2q"
server_key: "23bfabbd89e8c88a106133b66a23dc89"
server_key_iv: "68c8bb67513e985b38136384c4bb3f45"
server_key_version: "encryptionkey@0"
I20260807 19:43:12.715668 9267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "55b7387ad8f64bafab088fb1c76a6fc6"
format_stamp: "Formatted at 2026-08-07 19:43:12 on dist-test-slave-vq2q"
server_key: "23bfabbd89e8c88a106133b66a23dc89"
server_key_iv: "68c8bb67513e985b38136384c4bb3f45"
server_key_version: "encryptionkey@0"
I20260807 19:43:12.720671 9267 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:43:12.723955 9285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:12.725162 9267 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 19:43:12.725354 9267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "55b7387ad8f64bafab088fb1c76a6fc6"
format_stamp: "Formatted at 2026-08-07 19:43:12 on dist-test-slave-vq2q"
server_key: "23bfabbd89e8c88a106133b66a23dc89"
server_key_iv: "68c8bb67513e985b38136384c4bb3f45"
server_key_version: "encryptionkey@0"
I20260807 19:43:12.725546 9267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:12.737946 9267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:12.738749 9267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:12.739040 9267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:12.756510 9267 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:44521
I20260807 19:43:12.756528 9343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:44521 every 8 connection(s)
I20260807 19:43:12.757880 9267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:12.760223 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 9267
I20260807 19:43:12.760437 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:43:12.761059 31552 external_mini_cluster.cc:1428] Setting key 09958197a3c2e2a03a4b199c4009f6a3
I20260807 19:43:12.761801 9344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:12.775398 9344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6: Bootstrap starting.
I20260807 19:43:12.775399 9347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:12.763140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35872 (local address 127.30.208.62:44521)
0807 19:43:12.764119 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:12.764130 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:12.764170 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:43:12.766442 (+ 2272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:12.766466 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:12.766970 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:12.767301 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:12.767994 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:12.769340 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:12.771045 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:12.771659 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:12.771791 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:12.773043 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:12.773051 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:12.773064 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:12.773652 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:12.773732 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:12.773753 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:12.773878 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:12.774122 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 19:43:12.774422 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":353,"threads_started":1}
I20260807 19:43:12.779256 9344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:12.780221 9344 log.cc:824] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:12.782980 9344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6: No bootstrap required, opened a new log
I20260807 19:43:12.790692 9344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b7387ad8f64bafab088fb1c76a6fc6" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44521 } }
I20260807 19:43:12.791095 9344 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:12.791190 9344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55b7387ad8f64bafab088fb1c76a6fc6, State: Initialized, Role: FOLLOWER
I20260807 19:43:12.791796 9344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b7387ad8f64bafab088fb1c76a6fc6" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44521 } }
I20260807 19:43:12.792024 9344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:12.792160 9344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:12.792313 9344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:12.794809 9344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b7387ad8f64bafab088fb1c76a6fc6" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44521 } }
I20260807 19:43:12.795491 9344 leader_election.cc:302] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55b7387ad8f64bafab088fb1c76a6fc6; no voters:
I20260807 19:43:12.796351 9344 leader_election.cc:288] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:12.796553 9350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:12.797426 9350 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [term 1 LEADER]: Becoming Leader. State: Replica: 55b7387ad8f64bafab088fb1c76a6fc6, State: Running, Role: LEADER
I20260807 19:43:12.797998 9350 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b7387ad8f64bafab088fb1c76a6fc6" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44521 } }
I20260807 19:43:12.798736 9344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:12.801091 9352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55b7387ad8f64bafab088fb1c76a6fc6. Latest consensus state: current_term: 1 leader_uuid: "55b7387ad8f64bafab088fb1c76a6fc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b7387ad8f64bafab088fb1c76a6fc6" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44521 } } }
I20260807 19:43:12.801669 9352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:12.801818 9351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55b7387ad8f64bafab088fb1c76a6fc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b7387ad8f64bafab088fb1c76a6fc6" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44521 } } }
I20260807 19:43:12.802170 9351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6 [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:12.803464 9360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:12.809839 9360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:12.819298 9360 catalog_manager.cc:1347] Generated new cluster ID: bd4a4478c16f4d59871d108058741f8f
I20260807 19:43:12.819419 9360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:12.854614 9360 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:12.855926 9360 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:12.867265 9360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55b7387ad8f64bafab088fb1c76a6fc6: Generated new TSK 0
I20260807 19:43:12.868427 9360 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:43:12.882560 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44521
--builtin_ntp_servers=127.30.208.20:36959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 19:43:13.219126 9371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:13.219395 9371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:13.219503 9371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:13.227042 9371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:13.227154 9371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:13.227290 9371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:43:13.236696 9371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:43:13.238123 9371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:13.239856 9371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:13.252694 9376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:13.252787 9379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:13.252728 9377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:13.253638 9371 server_base.cc:1034] running on GCE node
I20260807 19:43:13.254241 9371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:13.255230 9371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:13.256435 9371 hybrid_clock.cc:648] HybridClock initialized: now 1786131793256376 us; error 60 us; skew 500 ppm
I20260807 19:43:13.256749 9371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:43:13.262919 9371 webserver.cc:460] Webserver started at http://127.30.208.1:37391/ using document root <none> and password file <none>
I20260807 19:43:13.263638 9371 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:13.263729 9371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:13.264046 9371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:13.266968 9371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "fef29619aa5e4962a3d9b9526ec94d02"
format_stamp: "Formatted at 2026-08-07 19:43:13 on dist-test-slave-vq2q"
server_key: "d67f7bc887c878f73e8914ae71cf6326"
server_key_iv: "02f9ba6705a85a8c1e2ed06d49494bb4"
server_key_version: "encryptionkey@0"
I20260807 19:43:13.267756 9371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "fef29619aa5e4962a3d9b9526ec94d02"
format_stamp: "Formatted at 2026-08-07 19:43:13 on dist-test-slave-vq2q"
server_key: "d67f7bc887c878f73e8914ae71cf6326"
server_key_iv: "02f9ba6705a85a8c1e2ed06d49494bb4"
server_key_version: "encryptionkey@0"
I20260807 19:43:13.273104 9371 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 19:43:13.276839 9386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:13.278617 9371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:43:13.278822 9371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "fef29619aa5e4962a3d9b9526ec94d02"
format_stamp: "Formatted at 2026-08-07 19:43:13 on dist-test-slave-vq2q"
server_key: "d67f7bc887c878f73e8914ae71cf6326"
server_key_iv: "02f9ba6705a85a8c1e2ed06d49494bb4"
server_key_version: "encryptionkey@0"
I20260807 19:43:13.279117 9371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:13.300462 9371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:13.301393 9371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:13.301676 9371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:13.302699 9371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:13.304217 9371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:13.304327 9371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:13.304435 9371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:13.304505 9371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:13.334084 9371 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:32887
I20260807 19:43:13.334137 9510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:32887 every 8 connection(s)
I20260807 19:43:13.335368 9371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:43:13.339860 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 9371
I20260807 19:43:13.340049 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:43:13.340637 31552 external_mini_cluster.cc:1428] Setting key fc5551e2ade252dd14a33e845be5490c
I20260807 19:43:13.343457 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44521
--builtin_ntp_servers=127.30.208.20:36959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 19:43:13.347711 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:13.337820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:40173 (local address 127.30.208.62:44521)
0807 19:43:13.338275 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:13.338280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:13.339316 (+ 1036us) server_negotiation.cc:407] Connection header received
0807 19:43:13.341164 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:13.341170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:13.341292 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:13.341388 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:13.342692 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:13.343413 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:13.344996 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:13.345460 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:13.345575 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:13.346893 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:13.346898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:13.346901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:13.346919 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:13.346958 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:13.346969 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:13.347044 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:13.347255 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 19:43:13.347353 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":235,"threads_started":1}
I20260807 19:43:13.348723 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:13.338271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:44521 (local address 127.30.208.1:40173)
0807 19:43:13.339152 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:13.339192 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:13.340899 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:13.341657 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:13.341669 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:13.341782 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:13.342525 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:13.342547 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:13.343564 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:13.343568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:13.344844 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:13.344858 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:13.345700 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:13.345709 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:13.345932 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:13.346570 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:43:13.346573 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:13.346612 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:43:13.346698 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:13.347058 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:13.347069 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:13.347077 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 19:43:13.347232 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 19:43:13.347646 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":266,"threads_started":1}
I20260807 19:43:13.350324 9511 heartbeater.cc:347] Connected to a master server at 127.30.208.62:44521
I20260807 19:43:13.350657 9511 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:13.351528 9511 heartbeater.cc:511] Master 127.30.208.62:44521 requested a full tablet report, sending...
I20260807 19:43:13.353852 9304 ts_manager.cc:194] Registered new tserver with Master: fef29619aa5e4962a3d9b9526ec94d02 (127.30.208.1:32887)
I20260807 19:43:13.355397 9304 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.208.1:40173
W20260807 19:43:13.680127 9516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:13.680419 9516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:13.680503 9516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:13.687719 9516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:13.687834 9516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:13.687942 9516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:43:13.697108 9516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:43:13.698385 9516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:13.700280 9516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:13.713596 9524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:13.713645 9522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:13.713622 9521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:13.714233 9516 server_base.cc:1034] running on GCE node
I20260807 19:43:13.714887 9516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:13.715932 9516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:13.717149 9516 hybrid_clock.cc:648] HybridClock initialized: now 1786131793717101 us; error 39 us; skew 500 ppm
I20260807 19:43:13.717522 9516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:43:13.719894 9516 webserver.cc:460] Webserver started at http://127.30.208.2:37825/ using document root <none> and password file <none>
I20260807 19:43:13.720623 9516 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:13.720721 9516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:13.721076 9516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:13.723954 9516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "4f57915ba3974f118ea0f48c831085b2"
format_stamp: "Formatted at 2026-08-07 19:43:13 on dist-test-slave-vq2q"
server_key: "f04ea1320749ea012aaae89ba35dfb8f"
server_key_iv: "527ceac42916cb5626778a34f74a3306"
server_key_version: "encryptionkey@0"
I20260807 19:43:13.724735 9516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "4f57915ba3974f118ea0f48c831085b2"
format_stamp: "Formatted at 2026-08-07 19:43:13 on dist-test-slave-vq2q"
server_key: "f04ea1320749ea012aaae89ba35dfb8f"
server_key_iv: "527ceac42916cb5626778a34f74a3306"
server_key_version: "encryptionkey@0"
I20260807 19:43:13.729935 9516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:43:13.733525 9530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:13.734962 9516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 19:43:13.735113 9516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "4f57915ba3974f118ea0f48c831085b2"
format_stamp: "Formatted at 2026-08-07 19:43:13 on dist-test-slave-vq2q"
server_key: "f04ea1320749ea012aaae89ba35dfb8f"
server_key_iv: "527ceac42916cb5626778a34f74a3306"
server_key_version: "encryptionkey@0"
I20260807 19:43:13.735322 9516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:13.751739 9516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:13.752779 9516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:13.753086 9516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:13.754020 9516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:13.755452 9516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:13.755538 9516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:13.755630 9516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:13.755710 9516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:13.795370 9516 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:34195
I20260807 19:43:13.795442 9655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:34195 every 8 connection(s)
I20260807 19:43:13.796617 9516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:43:13.802621 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 9516
I20260807 19:43:13.802780 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:43:13.803170 31552 external_mini_cluster.cc:1428] Setting key da648b182d63c02b0080c2b18977d1a5
I20260807 19:43:13.806138 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44521
--builtin_ntp_servers=127.30.208.20:36959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 19:43:13.809775 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:13.799044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:52715 (local address 127.30.208.62:44521)
0807 19:43:13.799213 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:13.799218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:13.800614 (+ 1396us) server_negotiation.cc:407] Connection header received
0807 19:43:13.802912 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:13.802918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:13.803000 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:13.803093 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:13.804404 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:13.805338 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:13.806947 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:13.807524 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:13.807650 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:13.809004 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:13.809010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:13.809014 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:13.809035 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:13.809073 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:13.809086 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:13.809167 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:13.809400 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 19:43:13.809496 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:43:13.810807 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:13.799528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:44521 (local address 127.30.208.2:52715)
0807 19:43:13.800460 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:13.800498 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:13.802115 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:13.803354 (+ 1239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:13.803364 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:13.803471 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:13.804202 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:13.804228 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:13.805481 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:13.805484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:13.806790 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:13.806803 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:13.807746 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:13.807752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:13.807985 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:13.808654 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:43:13.808657 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:13.808699 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:43:13.808781 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:13.809196 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:13.809206 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:13.809214 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 19:43:13.809375 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 19:43:13.809686 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":311,"threads_started":1}
I20260807 19:43:13.812211 9656 heartbeater.cc:347] Connected to a master server at 127.30.208.62:44521
I20260807 19:43:13.812498 9656 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:13.813318 9656 heartbeater.cc:511] Master 127.30.208.62:44521 requested a full tablet report, sending...
I20260807 19:43:13.814744 9304 ts_manager.cc:194] Registered new tserver with Master: 4f57915ba3974f118ea0f48c831085b2 (127.30.208.2:34195)
I20260807 19:43:13.815533 9304 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.208.2:52715
W20260807 19:43:14.139712 9659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:14.140007 9659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:14.140128 9659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:14.147341 9659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:14.147439 9659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:14.147576 9659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:43:14.156713 9659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:43:14.158021 9659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:14.159730 9659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:14.172080 9664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:14.172120 9665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:14.172224 9667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:14.173072 9659 server_base.cc:1034] running on GCE node
I20260807 19:43:14.173691 9659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:14.174662 9659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:14.175875 9659 hybrid_clock.cc:648] HybridClock initialized: now 1786131794175834 us; error 57 us; skew 500 ppm
I20260807 19:43:14.176182 9659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:43:14.178454 9659 webserver.cc:460] Webserver started at http://127.30.208.3:34601/ using document root <none> and password file <none>
I20260807 19:43:14.179127 9659 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:14.179232 9659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:14.179557 9659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:14.182437 9659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "c3b56df034e3459cb76a7de1b48e68f3"
format_stamp: "Formatted at 2026-08-07 19:43:14 on dist-test-slave-vq2q"
server_key: "cf295f5f24fb094f50d678d3b3f35e3c"
server_key_iv: "998e3d59b6708b910dbdce52fa4073ec"
server_key_version: "encryptionkey@0"
I20260807 19:43:14.183218 9659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "c3b56df034e3459cb76a7de1b48e68f3"
format_stamp: "Formatted at 2026-08-07 19:43:14 on dist-test-slave-vq2q"
server_key: "cf295f5f24fb094f50d678d3b3f35e3c"
server_key_iv: "998e3d59b6708b910dbdce52fa4073ec"
server_key_version: "encryptionkey@0"
I20260807 19:43:14.188405 9659 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:43:14.191749 9673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:14.192976 9659 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:43:14.193135 9659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "c3b56df034e3459cb76a7de1b48e68f3"
format_stamp: "Formatted at 2026-08-07 19:43:14 on dist-test-slave-vq2q"
server_key: "cf295f5f24fb094f50d678d3b3f35e3c"
server_key_iv: "998e3d59b6708b910dbdce52fa4073ec"
server_key_version: "encryptionkey@0"
I20260807 19:43:14.193382 9659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:14.212399 9659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:14.213320 9659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:14.213619 9659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:14.214601 9659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:14.216063 9659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:14.216166 9659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:14.216270 9659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:14.216353 9659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:14.247408 9659 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:35127
I20260807 19:43:14.247464 9796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:35127 every 8 connection(s)
I20260807 19:43:14.248795 9659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:43:14.252803 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 9659
I20260807 19:43:14.253012 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:43:14.253611 31552 external_mini_cluster.cc:1428] Setting key e50375750ed123657afc52f999d97416
I20260807 19:43:14.262009 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:14.251252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:49643 (local address 127.30.208.62:44521)
0807 19:43:14.251415 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:14.251420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:14.253135 (+ 1715us) server_negotiation.cc:407] Connection header received
0807 19:43:14.255469 (+ 2334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:14.255475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:14.255567 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:14.255668 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:14.256965 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:14.257808 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:14.259395 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:14.259853 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:14.259950 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:14.261295 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:14.261300 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:14.261303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:14.261321 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:14.261357 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:14.261369 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:14.261444 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:14.261642 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 19:43:14.261766 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:43:14.262892 9799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:14.251736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:44521 (local address 127.30.208.3:49643)
0807 19:43:14.252924 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:14.252981 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:14.255151 (+ 2170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:14.255971 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:14.255980 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:14.256074 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:14.256752 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:14.256779 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:14.257959 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:14.257962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:14.259250 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:14.259267 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:14.260056 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:14.260061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:14.260260 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:14.260946 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:43:14.260948 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:14.260989 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:43:14.261066 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:14.261451 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:14.261458 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:14.261463 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 19:43:14.261602 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 19:43:14.261901 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":301,"threads_started":1}
I20260807 19:43:14.264176 9797 heartbeater.cc:347] Connected to a master server at 127.30.208.62:44521
I20260807 19:43:14.264432 9797 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:14.265213 9797 heartbeater.cc:511] Master 127.30.208.62:44521 requested a full tablet report, sending...
I20260807 19:43:14.266605 9304 ts_manager.cc:194] Registered new tserver with Master: c3b56df034e3459cb76a7de1b48e68f3 (127.30.208.3:35127)
I20260807 19:43:14.267508 9304 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.208.3:49643
I20260807 19:43:14.269068 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 19:43:14.275051 9513 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:14.273448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35876 (local address 127.30.208.62:44521)
0807 19:43:14.273600 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:14.273604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:14.274109 (+ 505us) server_negotiation.cc:407] Connection header received
0807 19:43:14.274276 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:14.274282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:14.274366 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:14.274450 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:14.274869 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35876: BlockingRecv error: recv got EOF from 127.0.0.1:35876 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 19:43:14.275763 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 9371
I20260807 19:43:14.284384 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:14.276689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49770 (local address 127.30.208.1:32887)
0807 19:43:14.277270 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:14.277278 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:14.277304 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:43:14.277360 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:14.277366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:14.277575 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:14.277695 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:14.278113 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:14.279121 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:14.280711 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:14.281657 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:14.281795 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:14.282956 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:14.282965 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:14.282977 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:14.283579 (+ 602us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:14.283629 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:14.283650 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:14.283739 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:14.283871 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 19:43:14.284000 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":277,"threads_started":1}
I20260807 19:43:14.285687 9506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:14.365362 9511 heartbeater.cc:503] Master 127.30.208.62:44521 was elected leader, sending a full tablet report...
I20260807 19:43:14.369370 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 9371
I20260807 19:43:14.387420 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 9516
I20260807 19:43:14.394881 9808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:14.388283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57098 (local address 127.30.208.2:34195)
0807 19:43:14.388791 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:14.388799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:14.388827 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:43:14.388923 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:14.388930 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:14.389142 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:14.389307 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:14.389673 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:14.390672 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:14.391869 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:14.392395 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:14.392487 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:14.393490 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:14.393499 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:14.393511 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:14.394088 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:14.394141 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:14.394161 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:14.394247 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:14.394354 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 19:43:14.394489 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":239,"threads_started":1}
I20260807 19:43:14.396100 9651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:14.476796 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 9516
I20260807 19:43:14.492744 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 9659
I20260807 19:43:14.501302 9810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:14.493740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56956 (local address 127.30.208.3:35127)
0807 19:43:14.494466 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:14.494473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:14.494503 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:43:14.494567 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:14.494573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:14.494825 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:14.494962 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:14.495375 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:14.496444 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:14.497909 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:14.498529 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:14.498632 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:14.499762 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:14.499773 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:14.499787 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:14.500400 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:14.500450 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:14.500471 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:14.500582 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:14.500766 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 19:43:14.500924 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":385,"threads_started":1}
I20260807 19:43:14.502610 9791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:14.584447 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 9659
I20260807 19:43:14.600556 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 9267
I20260807 19:43:14.601838 9339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:14.706037 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 9267
2026-08-07T19:43:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2444 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T19:43:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:14Z Disabled control of system clock
I20260807 19:43:14.784651 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:41097
--webserver_interface=127.30.208.62
--webserver_port=0
--builtin_ntp_servers=127.30.208.20:33761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:41097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 19:43:15.118525 9816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:15.118827 9816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:15.118970 9816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:15.126118 9816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:15.126228 9816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:15.126287 9816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:15.126374 9816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:15.126434 9816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:15.135598 9816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:33761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:41097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:41097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:43:15.136905 9816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:15.138695 9816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:15.150166 9823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:15.150130 9821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:15.150602 9825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:15.151230 9816 server_base.cc:1034] running on GCE node
I20260807 19:43:15.152046 9816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:15.153513 9816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:15.154721 9816 hybrid_clock.cc:648] HybridClock initialized: now 1786131795154673 us; error 44 us; skew 500 ppm
I20260807 19:43:15.155042 9816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:43:15.157253 9816 webserver.cc:460] Webserver started at http://127.30.208.62:44141/ using document root <none> and password file <none>
I20260807 19:43:15.157935 9816 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:15.158041 9816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:15.158366 9816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:15.161263 9816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "454a9792211b4c28882da25f58db786d"
format_stamp: "Formatted at 2026-08-07 19:43:15 on dist-test-slave-vq2q"
server_key: "7ab58b1d6f7ed70ea537a7b14546cb7a"
server_key_iv: "fbab1e4ea1b1f43a400dcd0e53a7f004"
server_key_version: "encryptionkey@0"
I20260807 19:43:15.162042 9816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "454a9792211b4c28882da25f58db786d"
format_stamp: "Formatted at 2026-08-07 19:43:15 on dist-test-slave-vq2q"
server_key: "7ab58b1d6f7ed70ea537a7b14546cb7a"
server_key_iv: "fbab1e4ea1b1f43a400dcd0e53a7f004"
server_key_version: "encryptionkey@0"
I20260807 19:43:15.167124 9816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:43:15.170593 9832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:15.171914 9816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:43:15.172084 9816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "454a9792211b4c28882da25f58db786d"
format_stamp: "Formatted at 2026-08-07 19:43:15 on dist-test-slave-vq2q"
server_key: "7ab58b1d6f7ed70ea537a7b14546cb7a"
server_key_iv: "fbab1e4ea1b1f43a400dcd0e53a7f004"
server_key_version: "encryptionkey@0"
I20260807 19:43:15.172307 9816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:15.178117 9816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:15.178916 9816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:15.179198 9816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:15.196782 9816 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:41097
I20260807 19:43:15.196805 9889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:41097 every 8 connection(s)
I20260807 19:43:15.198156 9816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:15.200654 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 9816
I20260807 19:43:15.200837 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:43:15.201498 31552 external_mini_cluster.cc:1428] Setting key 509fa1374554fd248f1d8d9b6f6ce150
I20260807 19:43:15.202032 9890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:15.210147 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:15.203503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56024 (local address 127.30.208.62:41097)
0807 19:43:15.204497 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:15.204507 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:15.204552 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:43:15.206605 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:15.206628 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:15.207058 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:15.207408 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:15.208541 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:15.208548 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:15.208563 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:15.208581 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:15.208648 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:15.208664 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:15.208856 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:15.208957 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 19:43:15.209270 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":367,"threads_started":1}
I20260807 19:43:15.214920 9890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d: Bootstrap starting.
I20260807 19:43:15.217872 9890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:15.218886 9890 log.cc:824] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:15.221529 9890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d: No bootstrap required, opened a new log
I20260807 19:43:15.229025 9890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454a9792211b4c28882da25f58db786d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41097 } }
I20260807 19:43:15.229440 9890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:15.229532 9890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 454a9792211b4c28882da25f58db786d, State: Initialized, Role: FOLLOWER
I20260807 19:43:15.230070 9890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454a9792211b4c28882da25f58db786d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41097 } }
I20260807 19:43:15.230297 9890 raft_consensus.cc:388] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:15.230432 9890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:15.230587 9890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:15.232712 9890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454a9792211b4c28882da25f58db786d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41097 } }
I20260807 19:43:15.233268 9890 leader_election.cc:302] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 454a9792211b4c28882da25f58db786d; no voters:
I20260807 19:43:15.233855 9890 leader_election.cc:288] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:15.233974 9895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:15.235380 9895 raft_consensus.cc:686] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [term 1 LEADER]: Becoming Leader. State: Replica: 454a9792211b4c28882da25f58db786d, State: Running, Role: LEADER
I20260807 19:43:15.236120 9890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:15.236025 9895 consensus_queue.cc:237] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454a9792211b4c28882da25f58db786d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41097 } }
I20260807 19:43:15.238734 9896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "454a9792211b4c28882da25f58db786d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454a9792211b4c28882da25f58db786d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41097 } } }
I20260807 19:43:15.239058 9896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:15.239302 9897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 454a9792211b4c28882da25f58db786d. Latest consensus state: current_term: 1 leader_uuid: "454a9792211b4c28882da25f58db786d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "454a9792211b4c28882da25f58db786d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41097 } } }
I20260807 19:43:15.239631 9897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:15.241091 9903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:15.246500 9903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:15.255990 9903 catalog_manager.cc:1347] Generated new cluster ID: 31b69160552142238b431d975a8d852e
I20260807 19:43:15.256120 9903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:15.279639 9903 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:15.280164 9903 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:15.288470 9903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 454a9792211b4c28882da25f58db786d: Generated new TSK 0
I20260807 19:43:15.289325 9903 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:43:15.295565 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41097
--builtin_ntp_servers=127.30.208.20:33761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 19:43:15.660848 9914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:15.661131 9914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:15.661267 9914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:15.668480 9914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:15.668589 9914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:15.668733 9914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:43:15.678160 9914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:33761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:43:15.679472 9914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:15.681295 9914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:15.694448 9919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:15.694494 9922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:15.694410 9920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:15.695571 9914 server_base.cc:1034] running on GCE node
I20260807 19:43:15.696189 9914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:15.697211 9914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:15.698422 9914 hybrid_clock.cc:648] HybridClock initialized: now 1786131795698375 us; error 37 us; skew 500 ppm
I20260807 19:43:15.698746 9914 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:43:15.701007 9914 webserver.cc:460] Webserver started at http://127.30.208.1:35597/ using document root <none> and password file <none>
I20260807 19:43:15.701745 9914 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:15.701848 9914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:15.702162 9914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:15.707182 9914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "9071eeb9fc5f4ed3a50a743c802f9d52"
format_stamp: "Formatted at 2026-08-07 19:43:15 on dist-test-slave-vq2q"
server_key: "babc9369bf1cf9940c835179654a6ed9"
server_key_iv: "f632822ab8983c26ee369ed09860c268"
server_key_version: "encryptionkey@0"
I20260807 19:43:15.708019 9914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "9071eeb9fc5f4ed3a50a743c802f9d52"
format_stamp: "Formatted at 2026-08-07 19:43:15 on dist-test-slave-vq2q"
server_key: "babc9369bf1cf9940c835179654a6ed9"
server_key_iv: "f632822ab8983c26ee369ed09860c268"
server_key_version: "encryptionkey@0"
I20260807 19:43:15.713555 9914 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:43:15.717059 9928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:15.718500 9914 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:43:15.718655 9914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "9071eeb9fc5f4ed3a50a743c802f9d52"
format_stamp: "Formatted at 2026-08-07 19:43:15 on dist-test-slave-vq2q"
server_key: "babc9369bf1cf9940c835179654a6ed9"
server_key_iv: "f632822ab8983c26ee369ed09860c268"
server_key_version: "encryptionkey@0"
I20260807 19:43:15.718937 9914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:15.724638 9914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:15.725499 9914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:15.725776 9914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:15.726718 9914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:15.728176 9914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:15.728264 9914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:15.728375 9914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:15.728448 9914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:15.759778 9914 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:43843
I20260807 19:43:15.759866 10053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:43843 every 8 connection(s)
I20260807 19:43:15.761147 9914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:43:15.767887 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 9914
I20260807 19:43:15.768074 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:43:15.768815 31552 external_mini_cluster.cc:1428] Setting key 9096b9439536d3be26a97b534f6044f3
I20260807 19:43:15.769666 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:15.763812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:36681 (local address 127.30.208.62:41097)
0807 19:43:15.764308 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:15.764312 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:15.765543 (+ 1231us) server_negotiation.cc:407] Connection header received
0807 19:43:15.767263 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:15.767269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:15.767338 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:15.767452 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:15.768817 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:15.768824 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:15.768828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:15.768869 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:15.768927 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:15.768944 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:15.769060 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:15.769285 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 19:43:15.769355 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":241,"threads_started":1}
I20260807 19:43:15.770663 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:15.764382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:41097 (local address 127.30.208.1:36681)
0807 19:43:15.765364 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:15.765410 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:15.766988 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:15.767708 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:15.767716 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:15.767807 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:15.768514 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:43:15.768517 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:15.768555 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:43:15.768642 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:15.769048 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:15.769059 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:15.769068 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 19:43:15.769382 (+ 314us) client_negotiation.cc:240] Negotiation successful
0807 19:43:15.769648 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":283,"threads_started":1}
I20260807 19:43:15.771757 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41097
--builtin_ntp_servers=127.30.208.20:33761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 19:43:15.772274 10054 heartbeater.cc:347] Connected to a master server at 127.30.208.62:41097
I20260807 19:43:15.772581 10054 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:15.773238 10054 heartbeater.cc:511] Master 127.30.208.62:41097 requested a full tablet report, sending...
I20260807 19:43:15.775589 9851 ts_manager.cc:194] Registered new tserver with Master: 9071eeb9fc5f4ed3a50a743c802f9d52 (127.30.208.1:43843)
W20260807 19:43:16.109887 10059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:16.110160 10059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:16.110296 10059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:16.117467 10059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:16.117570 10059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:16.117722 10059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:43:16.126941 10059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:33761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:43:16.128201 10059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:16.129976 10059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:16.142354 10064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:16.142374 10065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:16.142995 10067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:16.143440 10059 server_base.cc:1034] running on GCE node
I20260807 19:43:16.143987 10059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:16.144892 10059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:16.146096 10059 hybrid_clock.cc:648] HybridClock initialized: now 1786131796146055 us; error 35 us; skew 500 ppm
I20260807 19:43:16.146456 10059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:43:16.148579 10059 webserver.cc:460] Webserver started at http://127.30.208.2:40451/ using document root <none> and password file <none>
I20260807 19:43:16.149319 10059 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:16.149406 10059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:16.149669 10059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:16.152417 10059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "fe771be258624cfe8299f39a50093ae5"
format_stamp: "Formatted at 2026-08-07 19:43:16 on dist-test-slave-vq2q"
server_key: "00e012ae312a4d24a4661b92cafaf39c"
server_key_iv: "4e7949cadcb117ee5334300f6c015d82"
server_key_version: "encryptionkey@0"
I20260807 19:43:16.153153 10059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "fe771be258624cfe8299f39a50093ae5"
format_stamp: "Formatted at 2026-08-07 19:43:16 on dist-test-slave-vq2q"
server_key: "00e012ae312a4d24a4661b92cafaf39c"
server_key_iv: "4e7949cadcb117ee5334300f6c015d82"
server_key_version: "encryptionkey@0"
I20260807 19:43:16.158187 10059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:43:16.161680 10075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:16.162997 10059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:43:16.163141 10059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "fe771be258624cfe8299f39a50093ae5"
format_stamp: "Formatted at 2026-08-07 19:43:16 on dist-test-slave-vq2q"
server_key: "00e012ae312a4d24a4661b92cafaf39c"
server_key_iv: "4e7949cadcb117ee5334300f6c015d82"
server_key_version: "encryptionkey@0"
I20260807 19:43:16.163345 10059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:16.168769 10059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:16.169591 10059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:16.169832 10059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:16.170727 10059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:16.172099 10059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:16.172186 10059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:16.172293 10059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:16.172364 10059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:16.205644 10059 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:43149
I20260807 19:43:16.205694 10200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:43149 every 8 connection(s)
I20260807 19:43:16.207211 10059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:43:16.210359 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 10059
I20260807 19:43:16.210515 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:43:16.211015 31552 external_mini_cluster.cc:1428] Setting key 2aca38841b00670e8e4c31b8e0d0d9b6
I20260807 19:43:16.214175 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41097
--builtin_ntp_servers=127.30.208.20:33761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 19:43:16.216214 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:16.209764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:46479 (local address 127.30.208.62:41097)
0807 19:43:16.209968 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:16.209972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:16.211731 (+ 1759us) server_negotiation.cc:407] Connection header received
0807 19:43:16.213744 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:16.213750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:16.213827 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:16.213935 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:16.215422 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:16.215428 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:16.215433 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:16.215454 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:16.215511 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:16.215526 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:16.215640 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:16.215869 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 19:43:16.215928 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 19:43:16.217430 10203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:16.210437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:41097 (local address 127.30.208.2:46479)
0807 19:43:16.211539 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:16.211601 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:16.213454 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:16.214190 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:16.214202 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:16.214335 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:16.215089 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:43:16.215092 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:16.215141 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:43:16.215248 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:16.215623 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:16.215632 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:16.215640 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 19:43:16.215923 (+ 283us) client_negotiation.cc:240] Negotiation successful
0807 19:43:16.216172 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":321,"threads_started":1}
I20260807 19:43:16.218662 10201 heartbeater.cc:347] Connected to a master server at 127.30.208.62:41097
I20260807 19:43:16.218947 10201 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:16.219524 10201 heartbeater.cc:511] Master 127.30.208.62:41097 requested a full tablet report, sending...
I20260807 19:43:16.220830 9851 ts_manager.cc:194] Registered new tserver with Master: fe771be258624cfe8299f39a50093ae5 (127.30.208.2:43149)
W20260807 19:43:16.559146 10205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:16.559449 10205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:16.559579 10205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:16.567130 10205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:16.567252 10205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:16.567400 10205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:43:16.576859 10205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:33761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:43:16.578174 10205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:16.579950 10205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:16.592883 10210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:16.592949 10213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:16.592923 10211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:16.593672 10205 server_base.cc:1034] running on GCE node
I20260807 19:43:16.594244 10205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:16.595155 10205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:16.596372 10205 hybrid_clock.cc:648] HybridClock initialized: now 1786131796596327 us; error 35 us; skew 500 ppm
I20260807 19:43:16.596678 10205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:43:16.598873 10205 webserver.cc:460] Webserver started at http://127.30.208.3:35669/ using document root <none> and password file <none>
I20260807 19:43:16.599557 10205 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:16.599648 10205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:16.599951 10205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:16.602819 10205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "be22c3ab0a5742f6b3664a255bec9b40"
format_stamp: "Formatted at 2026-08-07 19:43:16 on dist-test-slave-vq2q"
server_key: "5be6911bcda0fda4eeb9f75281381e26"
server_key_iv: "a779ca38733d793a5c99cbeb199cae8b"
server_key_version: "encryptionkey@0"
I20260807 19:43:16.603607 10205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "be22c3ab0a5742f6b3664a255bec9b40"
format_stamp: "Formatted at 2026-08-07 19:43:16 on dist-test-slave-vq2q"
server_key: "5be6911bcda0fda4eeb9f75281381e26"
server_key_iv: "a779ca38733d793a5c99cbeb199cae8b"
server_key_version: "encryptionkey@0"
I20260807 19:43:16.608853 10205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 19:43:16.612316 10219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:16.613773 10205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:43:16.613984 10205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "be22c3ab0a5742f6b3664a255bec9b40"
format_stamp: "Formatted at 2026-08-07 19:43:16 on dist-test-slave-vq2q"
server_key: "5be6911bcda0fda4eeb9f75281381e26"
server_key_iv: "a779ca38733d793a5c99cbeb199cae8b"
server_key_version: "encryptionkey@0"
I20260807 19:43:16.614212 10205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:16.619935 10205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:16.620719 10205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:16.621014 10205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:16.622085 10205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:16.623591 10205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:16.623689 10205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:16.623796 10205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:16.623873 10205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:16.654809 10205 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:42451
I20260807 19:43:16.654892 10347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:42451 every 8 connection(s)
I20260807 19:43:16.656142 10205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:43:16.662024 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 10205
I20260807 19:43:16.662205 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:43:16.662657 31552 external_mini_cluster.cc:1428] Setting key 71ccbb31e78ad78ec493dd78ab12340c
I20260807 19:43:16.664508 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:16.658697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:59753 (local address 127.30.208.62:41097)
0807 19:43:16.658871 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:16.658876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:16.660201 (+ 1325us) server_negotiation.cc:407] Connection header received
0807 19:43:16.662057 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:16.662062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:16.662138 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:16.662252 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:16.663683 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:16.663690 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:16.663695 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:16.663713 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:16.663748 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:16.663764 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:16.663869 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:16.664133 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 19:43:16.664204 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:43:16.665542 10350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:16.659149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:41097 (local address 127.30.208.3:59753)
0807 19:43:16.660029 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:16.660072 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:16.661731 (+ 1659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:16.662497 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:16.662505 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:16.662597 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:16.663319 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:43:16.663323 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:16.663377 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:43:16.663499 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:16.663859 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:16.663869 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:16.663878 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 19:43:16.664203 (+ 325us) client_negotiation.cc:240] Negotiation successful
0807 19:43:16.664421 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":266,"threads_started":1}
I20260807 19:43:16.666875 10348 heartbeater.cc:347] Connected to a master server at 127.30.208.62:41097
I20260807 19:43:16.667165 10348 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:16.667835 10348 heartbeater.cc:511] Master 127.30.208.62:41097 requested a full tablet report, sending...
I20260807 19:43:16.669216 9851 ts_manager.cc:194] Registered new tserver with Master: be22c3ab0a5742f6b3664a255bec9b40 (127.30.208.3:42451)
I20260807 19:43:16.678442 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 19:43:16.684659 10056 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:43:16.683052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56028 (local address 127.30.208.62:41097)
0807 19:43:16.683207 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:16.683211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:16.683712 (+ 501us) server_negotiation.cc:407] Connection header received
0807 19:43:16.683891 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:16.683894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:16.683951 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:16.684043 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:16.684399 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56028: BlockingRecv error: recv got EOF from 127.0.0.1:56028 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:43:16.685369 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 9914
I20260807 19:43:16.688712 10357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:16.686352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41644 (local address 127.30.208.1:43843)
0807 19:43:16.686867 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:16.686873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:16.686900 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:43:16.686957 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:16.686963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:16.687166 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:16.687282 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:16.688117 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:16.688124 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:16.688135 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:16.688154 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:16.688196 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:16.688210 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:16.688315 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:16.688401 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 19:43:16.688480 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":232,"threads_started":1}
I20260807 19:43:16.689922 10049 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:16.771139 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 9914
I20260807 19:43:16.787587 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 10059
I20260807 19:43:16.790894 10359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:16.788440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55044 (local address 127.30.208.2:43149)
0807 19:43:16.788964 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:16.788971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:16.788999 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:43:16.789064 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:16.789070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:16.789327 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:16.789459 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:16.790274 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:16.790281 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:16.790292 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:16.790305 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:16.790353 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:16.790371 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:16.790477 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:16.790552 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 19:43:16.790636 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":230,"threads_started":1}
I20260807 19:43:16.792091 10196 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:16.873317 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 10059
I20260807 19:43:16.889961 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 10205
I20260807 19:43:16.893941 10361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:16.891028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40990 (local address 127.30.208.3:42451)
0807 19:43:16.891731 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:16.891738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:16.891770 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:43:16.891834 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:16.891868 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:16.892087 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:16.892221 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:16.893154 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:16.893160 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:16.893171 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:43:16.893217 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:16.893270 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:43:16.893288 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:16.893434 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:16.893491 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 19:43:16.893586 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":377,"threads_started":1}
I20260807 19:43:16.895172 10342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:16.977864 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 10205
I20260807 19:43:16.994503 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 9816
I20260807 19:43:16.995777 9884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:17.098174 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 9816
2026-08-07T19:43:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2393 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:17 dist-test-slave-vq2q krb5kdc[10366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:17 dist-test-slave-vq2q krb5kdc[10366](info): set up 2 sockets
Aug 07 19:43:17 dist-test-slave-vq2q krb5kdc[10366](info): commencing operation
krb5kdc: starting...
W20260807 19:43:18.180155 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:43:18 dist-test-slave-vq2q krb5kdc[10366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:43:18.461417 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:45231
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:45231
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:18.796977 10379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:18.797293 10379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:18.797384 10379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:18.804627 10379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:43:18.804754 10379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:18.804800 10379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:18.804863 10379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:43:18.804908 10379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:43:18.814834 10379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:45231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:45231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:43:18.816146 10379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:18.817543 10379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:18.829250 10384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:18.829205 10387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:18.829283 10385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:18.834685 10379 server_base.cc:1034] running on GCE node
I20260807 19:43:18.835510 10379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:18.836948 10379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:18.838155 10379 hybrid_clock.cc:648] HybridClock initialized: now 1786131798838095 us; error 42 us; skew 500 ppm
I20260807 19:43:18.838515 10379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:18 dist-test-slave-vq2q krb5kdc[10366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131798, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:18.846196 10379 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:43:18.847842 10379 webserver.cc:460] Webserver started at http://127.30.208.62:38261/ using document root <none> and password file <none>
I20260807 19:43:18.848572 10379 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:18.848698 10379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:18.849020 10379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:18.852125 10379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "673952fd98fa4292ae661702432d5b8d"
format_stamp: "Formatted at 2026-08-07 19:43:18 on dist-test-slave-vq2q"
server_key: "3ce0810c4a12ca2f378e52deb8d46387"
server_key_iv: "2fc26c5129dec9a9786a4b30d97dc0d0"
server_key_version: "encryptionkey@0"
I20260807 19:43:18.852895 10379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "673952fd98fa4292ae661702432d5b8d"
format_stamp: "Formatted at 2026-08-07 19:43:18 on dist-test-slave-vq2q"
server_key: "3ce0810c4a12ca2f378e52deb8d46387"
server_key_iv: "2fc26c5129dec9a9786a4b30d97dc0d0"
server_key_version: "encryptionkey@0"
I20260807 19:43:18.858217 10379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:43:18.861573 10398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:18.862893 10379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:43:18.863051 10379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "673952fd98fa4292ae661702432d5b8d"
format_stamp: "Formatted at 2026-08-07 19:43:18 on dist-test-slave-vq2q"
server_key: "3ce0810c4a12ca2f378e52deb8d46387"
server_key_iv: "2fc26c5129dec9a9786a4b30d97dc0d0"
server_key_version: "encryptionkey@0"
I20260807 19:43:18.863227 10379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:18.902266 10379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:18.906548 10379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:18.906837 10379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:18.924978 10379 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:45231
I20260807 19:43:18.925009 10458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:45231 every 8 connection(s)
I20260807 19:43:18.926420 10379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:43:18.928726 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 10379
I20260807 19:43:18.928953 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:43:18.929795 31552 external_mini_cluster.cc:1428] Setting key 16caab266038e0051da478f492fe49ad
I20260807 19:43:18.930832 10459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:43:18 dist-test-slave-vq2q krb5kdc[10366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:43:18.945570 10459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d: Bootstrap starting.
I20260807 19:43:18.948836 10459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:18.949910 10459 log.cc:824] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:18.952772 10459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d: No bootstrap required, opened a new log
I20260807 19:43:18.952917 10462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:18.931900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59782 (local address 127.30.208.62:45231)
0807 19:43:18.932670 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:18.932682 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:18.932723 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:43:18.934303 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:18.934325 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:18.934698 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:18.935041 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:18.936989 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:18.938402 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:18.940138 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:18.940866 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:18.941012 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:18.945954 (+ 4942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:18.945964 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:18.945980 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:18.946025 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:18.948903 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:18.949672 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:18.949679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:18.949685 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:18.949866 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:18.950259 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:18.950263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:18.950265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:18.950803 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:18.951133 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:18.951523 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 19:43:18.951935 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":349,"threads_started":1}
I20260807 19:43:18.960865 10459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "673952fd98fa4292ae661702432d5b8d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45231 } }
I20260807 19:43:18.961282 10459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:18.961383 10459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 673952fd98fa4292ae661702432d5b8d, State: Initialized, Role: FOLLOWER
I20260807 19:43:18.961921 10459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "673952fd98fa4292ae661702432d5b8d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45231 } }
I20260807 19:43:18.962165 10459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:43:18.962324 10459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:43:18.962491 10459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:18.964692 10459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "673952fd98fa4292ae661702432d5b8d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45231 } }
I20260807 19:43:18.965250 10459 leader_election.cc:302] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 673952fd98fa4292ae661702432d5b8d; no voters:
I20260807 19:43:18.965935 10459 leader_election.cc:288] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:43:18.966077 10464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:18.967072 10464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [term 1 LEADER]: Becoming Leader. State: Replica: 673952fd98fa4292ae661702432d5b8d, State: Running, Role: LEADER
I20260807 19:43:18.967620 10464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "673952fd98fa4292ae661702432d5b8d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45231 } }
I20260807 19:43:18.968106 10459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:43:18.970357 10466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 673952fd98fa4292ae661702432d5b8d. Latest consensus state: current_term: 1 leader_uuid: "673952fd98fa4292ae661702432d5b8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "673952fd98fa4292ae661702432d5b8d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45231 } } }
I20260807 19:43:18.970325 10465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "673952fd98fa4292ae661702432d5b8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "673952fd98fa4292ae661702432d5b8d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 45231 } } }
I20260807 19:43:18.970832 10465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:18.970832 10466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d [sys.catalog]: This master's current role is: LEADER
I20260807 19:43:18.973428 10471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:43:18.978621 10471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:43:18.987531 10471 catalog_manager.cc:1347] Generated new cluster ID: e424396dd4c44eafa0ab6eb2f442a6d2
I20260807 19:43:18.987666 10471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:43:19.002628 10471 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:43:19.004593 10471 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:43:19.018270 10471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d: Generated new TSK 0
I20260807 19:43:19.019523 10471 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:43:19.093291 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45231
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:19.427826 10490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:19.428138 10490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:19.428275 10490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:19.435735 10490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:19.435881 10490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:19.436031 10490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:43:19.446051 10490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45231
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:43:19.447398 10490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:19.448735 10490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:19.461607 10495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:19.461690 10496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:19.462095 10499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:19.462489 10490 server_base.cc:1034] running on GCE node
I20260807 19:43:19.463084 10490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:19.464079 10490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:19.465320 10490 hybrid_clock.cc:648] HybridClock initialized: now 1786131799465262 us; error 41 us; skew 500 ppm
I20260807 19:43:19.465672 10490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:19 dist-test-slave-vq2q krb5kdc[10366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131799, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:19.468878 10490 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:43:19.470408 10490 webserver.cc:460] Webserver started at http://127.30.208.1:33621/ using document root <none> and password file <none>
I20260807 19:43:19.471088 10490 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:19.471210 10490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:19.471494 10490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:19.474428 10490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "1b33cacd844b4ec4beca1ca664f289af"
format_stamp: "Formatted at 2026-08-07 19:43:19 on dist-test-slave-vq2q"
server_key: "8b178fd48c54147ba20e71a904bfd22e"
server_key_iv: "80dd7e9de2274d112e5869f613cec3d1"
server_key_version: "encryptionkey@0"
I20260807 19:43:19.475162 10490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "1b33cacd844b4ec4beca1ca664f289af"
format_stamp: "Formatted at 2026-08-07 19:43:19 on dist-test-slave-vq2q"
server_key: "8b178fd48c54147ba20e71a904bfd22e"
server_key_iv: "80dd7e9de2274d112e5869f613cec3d1"
server_key_version: "encryptionkey@0"
I20260807 19:43:19.480425 10490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:43:19.483796 10507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:19.485069 10490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:43:19.485260 10490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "1b33cacd844b4ec4beca1ca664f289af"
format_stamp: "Formatted at 2026-08-07 19:43:19 on dist-test-slave-vq2q"
server_key: "8b178fd48c54147ba20e71a904bfd22e"
server_key_iv: "80dd7e9de2274d112e5869f613cec3d1"
server_key_version: "encryptionkey@0"
I20260807 19:43:19.485445 10490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:19.515903 10490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:19.520314 10490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:19.520610 10490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:19.521668 10490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:19.523038 10490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:19.523113 10490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:19.523216 10490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:19.523268 10490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:19.553370 10490 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:35727
I20260807 19:43:19.553424 10630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:35727 every 8 connection(s)
I20260807 19:43:19.554654 10490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:43:19.561830 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 10490
I20260807 19:43:19.562027 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:43:19.562667 31552 external_mini_cluster.cc:1428] Setting key a13da5fea67e3e5188245b832e95f804
Aug 07 19:43:19 dist-test-slave-vq2q krb5kdc[10366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131799, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:43:19.575073 10633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.557159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:55487 (local address 127.30.208.62:45231)
0807 19:43:19.557642 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:19.557647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:19.558596 (+ 949us) server_negotiation.cc:407] Connection header received
0807 19:43:19.560453 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:19.560459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:19.560543 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:19.560643 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:19.562375 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.563140 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.564797 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.565229 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.565345 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.569333 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:19.569339 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:19.569342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:19.569375 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:19.571906 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.572646 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.572652 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.572656 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.572796 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:19.573262 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:19.573267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:19.573270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:19.573561 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:19.573702 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:19.574457 (+ 755us) server_negotiation.cc:299] Negotiation successful
0807 19:43:19.574694 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":213,"threads_started":1}
I20260807 19:43:19.576071 10634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:19.557634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:45231 (local address 127.30.208.1:55487)
0807 19:43:19.558398 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:19.558447 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:19.560158 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:19.560923 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:19.560932 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:19.561548 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:19.562186 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.562207 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.563349 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.563354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.564638 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:19.564652 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:19.565448 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:19.565456 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:19.565738 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:19.566394 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:19.566417 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:19.569028 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:19.572058 (+ 3030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.572068 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.572086 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.572527 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.572909 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:19.572913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:19.572917 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:19.573117 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:19.573694 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:19.573702 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:19.574060 (+ 358us) client_negotiation.cc:769] Sending connection context
0807 19:43:19.574319 (+ 259us) client_negotiation.cc:240] Negotiation successful
0807 19:43:19.574867 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":316,"threads_started":1}
I20260807 19:43:19.577870 10631 heartbeater.cc:347] Connected to a master server at 127.30.208.62:45231
I20260807 19:43:19.578168 10631 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:19.578920 10631 heartbeater.cc:511] Master 127.30.208.62:45231 requested a full tablet report, sending...
I20260807 19:43:19.581271 10418 ts_manager.cc:194] Registered new tserver with Master: 1b33cacd844b4ec4beca1ca664f289af (127.30.208.1:35727)
I20260807 19:43:19.583256 10418 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:55487
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:43:19.624174 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45231
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:19.967589 10639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:19.967911 10639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:19.968061 10639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:19.975872 10639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:19.975998 10639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:19.976154 10639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:43:19.986508 10639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45231
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:43:19.987867 10639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:19.989218 10639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:20.001838 10646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:20.001909 10645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:20.002436 10648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:20.002849 10639 server_base.cc:1034] running on GCE node
I20260807 19:43:20.003515 10639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:20.004467 10639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:20.005676 10639 hybrid_clock.cc:648] HybridClock initialized: now 1786131800005608 us; error 55 us; skew 500 ppm
I20260807 19:43:20.006000 10639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:20 dist-test-slave-vq2q krb5kdc[10366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131800, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:20.009123 10639 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:43:20.010695 10639 webserver.cc:460] Webserver started at http://127.30.208.2:39329/ using document root <none> and password file <none>
I20260807 19:43:20.011386 10639 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:20.011497 10639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:20.011752 10639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:20.014899 10639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "3e7fb61ca4ba4cc7accc63645a66c117"
format_stamp: "Formatted at 2026-08-07 19:43:20 on dist-test-slave-vq2q"
server_key: "d11bf9a0873fbed1ba3fe21079df6e5e"
server_key_iv: "44860d94eb7965c830cde6c4bffa608c"
server_key_version: "encryptionkey@0"
I20260807 19:43:20.015662 10639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "3e7fb61ca4ba4cc7accc63645a66c117"
format_stamp: "Formatted at 2026-08-07 19:43:20 on dist-test-slave-vq2q"
server_key: "d11bf9a0873fbed1ba3fe21079df6e5e"
server_key_iv: "44860d94eb7965c830cde6c4bffa608c"
server_key_version: "encryptionkey@0"
I20260807 19:43:20.021281 10639 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:43:20.024883 10658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:20.026278 10639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 19:43:20.026451 10639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "3e7fb61ca4ba4cc7accc63645a66c117"
format_stamp: "Formatted at 2026-08-07 19:43:20 on dist-test-slave-vq2q"
server_key: "d11bf9a0873fbed1ba3fe21079df6e5e"
server_key_iv: "44860d94eb7965c830cde6c4bffa608c"
server_key_version: "encryptionkey@0"
I20260807 19:43:20.026641 10639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:20.047130 10639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:20.051468 10639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:20.051731 10639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:20.052673 10639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:20.054044 10639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:20.054126 10639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:20.054208 10639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:20.054253 10639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:20.085353 10639 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:43241
I20260807 19:43:20.085397 10781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:43241 every 8 connection(s)
I20260807 19:43:20.086656 10639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:43:20.092099 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 10639
I20260807 19:43:20.092253 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:43:20.092680 31552 external_mini_cluster.cc:1428] Setting key fb31d38aad1594fb9015c83a53f54474
Aug 07 19:43:20 dist-test-slave-vq2q krb5kdc[10366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131800, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:43:20.107786 10784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.089379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:58183 (local address 127.30.208.62:45231)
0807 19:43:20.089971 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.089977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.090963 (+ 986us) server_negotiation.cc:407] Connection header received
0807 19:43:20.092925 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.092931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.093017 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.093143 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.094906 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.095909 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.097503 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.098071 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.098195 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.101588 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:20.101593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:20.101597 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:20.101636 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:20.104584 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:20.105353 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:20.105359 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:20.105363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:20.105526 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:20.105948 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:20.105952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:20.105956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:20.106298 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:20.106461 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.107325 (+ 864us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.107515 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":330,"threads_started":1}
I20260807 19:43:20.109264 10785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.089925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:45231 (local address 127.30.208.2:58183)
0807 19:43:20.090801 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:20.090840 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:20.092586 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:20.093470 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:20.093482 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:20.094029 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:20.094700 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:20.094721 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.096064 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.096068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:20.097364 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:20.097378 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.098283 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.098288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:20.098481 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.099112 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:20.099140 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:20.101345 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:20.104730 (+ 3385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:20.104736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:20.104749 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:20.105200 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:20.105626 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:20.105629 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:20.105631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:20.105840 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:20.106463 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:20.106470 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:20.106944 (+ 474us) client_negotiation.cc:769] Sending connection context
0807 19:43:20.107236 (+ 292us) client_negotiation.cc:240] Negotiation successful
0807 19:43:20.107769 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":382,"threads_started":1}
I20260807 19:43:20.110733 10782 heartbeater.cc:347] Connected to a master server at 127.30.208.62:45231
I20260807 19:43:20.111022 10782 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:20.111747 10782 heartbeater.cc:511] Master 127.30.208.62:45231 requested a full tablet report, sending...
I20260807 19:43:20.113234 10418 ts_manager.cc:194] Registered new tserver with Master: 3e7fb61ca4ba4cc7accc63645a66c117 (127.30.208.2:43241)
I20260807 19:43:20.114053 10418 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:58183
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:43:20.153960 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45231
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:43:20.488646 10788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:43:20.488929 10788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:43:20.489053 10788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:43:20.496407 10788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:43:20.496541 10788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:43:20.496694 10788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:43:20.506785 10788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:45231
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:43:20.508126 10788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:43:20.509500 10788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:43:20.522248 10797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:20.522294 10795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:43:20.522331 10799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:43:20.523164 10788 server_base.cc:1034] running on GCE node
I20260807 19:43:20.523818 10788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:43:20.524767 10788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:43:20.525998 10788 hybrid_clock.cc:648] HybridClock initialized: now 1786131800525939 us; error 46 us; skew 500 ppm
I20260807 19:43:20.526321 10788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:43:20 dist-test-slave-vq2q krb5kdc[10366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131800, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:43:20.529598 10788 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:43:20.531220 10788 webserver.cc:460] Webserver started at http://127.30.208.3:41035/ using document root <none> and password file <none>
I20260807 19:43:20.531960 10788 fs_manager.cc:359] Metadata directory not provided
I20260807 19:43:20.532070 10788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:43:20.532362 10788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:43:20.535478 10788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "21eafb859c1d4847a218ba294c7fc667"
format_stamp: "Formatted at 2026-08-07 19:43:20 on dist-test-slave-vq2q"
server_key: "1a7bbc708549a750361c43913ab697bf"
server_key_iv: "816e9d071e17ef9b80b31a918096402d"
server_key_version: "encryptionkey@0"
I20260807 19:43:20.536223 10788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "21eafb859c1d4847a218ba294c7fc667"
format_stamp: "Formatted at 2026-08-07 19:43:20 on dist-test-slave-vq2q"
server_key: "1a7bbc708549a750361c43913ab697bf"
server_key_iv: "816e9d071e17ef9b80b31a918096402d"
server_key_version: "encryptionkey@0"
I20260807 19:43:20.541676 10788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:43:20.545104 10807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:20.546576 10788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:43:20.546767 10788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "21eafb859c1d4847a218ba294c7fc667"
format_stamp: "Formatted at 2026-08-07 19:43:20 on dist-test-slave-vq2q"
server_key: "1a7bbc708549a750361c43913ab697bf"
server_key_iv: "816e9d071e17ef9b80b31a918096402d"
server_key_version: "encryptionkey@0"
I20260807 19:43:20.546973 10788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:43:20.564368 10788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:43:20.569401 10788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:43:20.569677 10788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:43:20.570667 10788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:43:20.572050 10788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:43:20.572121 10788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:20.572252 10788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:43:20.572300 10788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:43:20.586362 10631 heartbeater.cc:503] Master 127.30.208.62:45231 was elected leader, sending a full tablet report...
I20260807 19:43:20.603580 10788 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:35003
I20260807 19:43:20.603621 10931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:35003 every 8 connection(s)
I20260807 19:43:20.604808 10788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:43:20.610774 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 10788
I20260807 19:43:20.610977 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:43:20.611565 31552 external_mini_cluster.cc:1428] Setting key 3051965aaf638d7a1c3669bb109cbd95
Aug 07 19:43:20 dist-test-slave-vq2q krb5kdc[10366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131800, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:43:20.624257 10784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.607348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:35853 (local address 127.30.208.62:45231)
0807 19:43:20.607518 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.607522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.608634 (+ 1112us) server_negotiation.cc:407] Connection header received
0807 19:43:20.610486 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.610490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.610574 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.610654 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.612564 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.613484 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.614985 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.615370 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.615483 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.618866 (+ 3383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:20.618871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:20.618874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:20.618908 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:20.621285 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:20.621981 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:20.621987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:20.621989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:20.622112 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:20.622537 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:20.622541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:20.622543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:20.622828 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:20.622986 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.623688 (+ 702us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.623937 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:43:20.625383 10934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.607780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:45231 (local address 127.30.208.3:35853)
0807 19:43:20.608480 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:20.608518 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:20.610182 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:20.610941 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:20.610952 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:20.611692 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:43:20.612391 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:20.612410 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.613641 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.613645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:20.614844 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:20.614857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.615585 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.615593 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:20.615809 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.616420 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:43:20.616442 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:43:20.618644 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:43:20.621417 (+ 2773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:20.621426 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:20.621445 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:20.621863 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:20.622218 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:43:20.622223 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:43:20.622226 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:43:20.622428 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:43:20.622969 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:43:20.622977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:43:20.623317 (+ 340us) client_negotiation.cc:769] Sending connection context
0807 19:43:20.623579 (+ 262us) client_negotiation.cc:240] Negotiation successful
0807 19:43:20.624121 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":270,"threads_started":1}
I20260807 19:43:20.627064 10932 heartbeater.cc:347] Connected to a master server at 127.30.208.62:45231
I20260807 19:43:20.627383 10932 heartbeater.cc:464] Registering TS with master...
I20260807 19:43:20.628079 10932 heartbeater.cc:511] Master 127.30.208.62:45231 requested a full tablet report, sending...
I20260807 19:43:20.629474 10418 ts_manager.cc:194] Registered new tserver with Master: 21eafb859c1d4847a218ba294c7fc667 (127.30.208.3:35003)
I20260807 19:43:20.630312 10418 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:35853
I20260807 19:43:20.638960 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:43:20.655319 10784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.643348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58870 (local address 127.30.208.62:45231)
0807 19:43:20.643567 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.643574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.643913 (+ 339us) server_negotiation.cc:407] Connection header received
0807 19:43:20.644054 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.644057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.644129 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.644205 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.645594 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.646327 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.647555 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.647966 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.648060 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.650364 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:20.650370 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:20.650373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:20.650403 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:20.652823 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:20.653598 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:20.653603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:20.653605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:20.653745 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:20.654057 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:20.654061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:20.654063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:20.654388 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:20.654593 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.654892 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.655065 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:43:20.659446 10418 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:43:20.666342 10418 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:43:20.701663 10951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.678732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:35727 (local address 127.0.0.1:57962)
0807 19:43:20.686552 (+ 7820us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:20.686574 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:20.686714 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:20.693550 (+ 6836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:20.693558 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:20.693589 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:20.694080 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:20.694094 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.696181 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.696186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:20.698754 (+ 2568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:20.698770 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.700512 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.700519 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:20.700720 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.700775 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 19:43:20.700919 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 19:43:20.701124 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7671,"mutex_wait_us":30,"thread_start_us":1082,"threads_started":1}
I20260807 19:43:20.702190 10949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.678460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:43241 (local address 127.0.0.1:49514)
0807 19:43:20.685596 (+ 7136us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:20.685678 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:20.685908 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:20.695349 (+ 9441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:20.695354 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:20.695381 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:20.695727 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:20.695736 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.697549 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.697555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:20.699565 (+ 2010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:20.699579 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.701341 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.701350 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:20.701534 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.701566 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:43:20.701642 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 19:43:20.701786 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6791,"mutex_wait_us":31,"thread_start_us":594,"threads_started":1}
I20260807 19:43:20.702354 10954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.680574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59158 (local address 127.30.208.3:35003)
0807 19:43:20.694370 (+ 13796us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.694379 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.694417 (+ 38us) server_negotiation.cc:407] Connection header received
0807 19:43:20.694516 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.694523 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.694797 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.695004 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:20.695633 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.696907 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.699364 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.700400 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.700526 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.700763 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.701836 (+ 1073us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.702007 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12520,"thread_start_us":12607,"threads_started":1}
I20260807 19:43:20.701713 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.680178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:35003 (local address 127.0.0.1:59158)
0807 19:43:20.693248 (+ 13070us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:20.693269 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:20.693415 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:20.695019 (+ 1604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:20.695023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:20.695049 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:20.695448 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:20.695460 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.697079 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.697085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:20.699185 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:20.699201 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.700903 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.700909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:20.701072 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.701103 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 19:43:20.701202 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 19:43:20.701346 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12918,"mutex_wait_us":141,"thread_start_us":13323,"threads_started":1}
I20260807 19:43:20.702586 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.679455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57962 (local address 127.30.208.1:35727)
0807 19:43:20.692718 (+ 13263us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.692730 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.692772 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:43:20.692920 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.692930 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.693268 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.693538 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:20.694294 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.696036 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.698934 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.700272 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.700432 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.702031 (+ 1599us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.702104 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.702274 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13044,"thread_start_us":14286,"threads_started":1}
I20260807 19:43:20.703184 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.678696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49514 (local address 127.30.208.2:43241)
0807 19:43:20.694363 (+ 15667us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.694372 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.694402 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:43:20.694504 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.694512 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.695101 (+ 589us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.695335 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:20.695894 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.697360 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.699729 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.701072 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.702498 (+ 1426us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.702689 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.702757 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.702913 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15470,"thread_start_us":15993,"threads_started":1}
I20260807 19:43:20.706660 10558 tablet_service.cc:1467] Processing CreateTablet for tablet 333f3ba8dbc44186b31fdda9c989fe2a (DEFAULT_TABLE table=test-table [id=e7aadb41146c400daa3f26bf68b3b439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:20.706658 10709 tablet_service.cc:1467] Processing CreateTablet for tablet 333f3ba8dbc44186b31fdda9c989fe2a (DEFAULT_TABLE table=test-table [id=e7aadb41146c400daa3f26bf68b3b439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:20.706658 10863 tablet_service.cc:1467] Processing CreateTablet for tablet 333f3ba8dbc44186b31fdda9c989fe2a (DEFAULT_TABLE table=test-table [id=e7aadb41146c400daa3f26bf68b3b439]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:43:20.708065 10558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 333f3ba8dbc44186b31fdda9c989fe2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:20.708099 10863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 333f3ba8dbc44186b31fdda9c989fe2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:20.708103 10709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 333f3ba8dbc44186b31fdda9c989fe2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:43:20.720023 10955 tablet_bootstrap.cc:492] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af: Bootstrap starting.
I20260807 19:43:20.722543 10956 tablet_bootstrap.cc:492] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667: Bootstrap starting.
I20260807 19:43:20.723364 10957 tablet_bootstrap.cc:492] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117: Bootstrap starting.
I20260807 19:43:20.723814 10955 tablet_bootstrap.cc:654] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:20.725235 10955 log.cc:824] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:20.726851 10956 tablet_bootstrap.cc:654] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:20.726868 10957 tablet_bootstrap.cc:654] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117: Neither blocks nor log segments found. Creating new log.
I20260807 19:43:20.727906 10956 log.cc:824] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:20.728060 10957 log.cc:824] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117: Log is configured to *not* fsync() on all Append() calls
I20260807 19:43:20.728899 10955 tablet_bootstrap.cc:492] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af: No bootstrap required, opened a new log
I20260807 19:43:20.729147 10955 ts_tablet_manager.cc:1397] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:43:20.730645 10957 tablet_bootstrap.cc:492] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117: No bootstrap required, opened a new log
I20260807 19:43:20.730909 10957 ts_tablet_manager.cc:1397] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:43:20.730926 10956 tablet_bootstrap.cc:492] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667: No bootstrap required, opened a new log
I20260807 19:43:20.731158 10956 ts_tablet_manager.cc:1397] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260807 19:43:20.738440 10957 raft_consensus.cc:348] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e7fb61ca4ba4cc7accc63645a66c117" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 43241 } } peers { permanent_uuid: "1b33cacd844b4ec4beca1ca664f289af" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35727 } } peers { permanent_uuid: "21eafb859c1d4847a218ba294c7fc667" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35003 } }
I20260807 19:43:20.738852 10957 raft_consensus.cc:374] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:20.738970 10957 raft_consensus.cc:729] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e7fb61ca4ba4cc7accc63645a66c117, State: Initialized, Role: FOLLOWER
I20260807 19:43:20.739622 10957 consensus_queue.cc:260] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e7fb61ca4ba4cc7accc63645a66c117" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 43241 } } peers { permanent_uuid: "1b33cacd844b4ec4beca1ca664f289af" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35727 } } peers { permanent_uuid: "21eafb859c1d4847a218ba294c7fc667" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35003 } }
I20260807 19:43:20.739763 10955 raft_consensus.cc:348] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e7fb61ca4ba4cc7accc63645a66c117" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 43241 } } peers { permanent_uuid: "1b33cacd844b4ec4beca1ca664f289af" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35727 } } peers { permanent_uuid: "21eafb859c1d4847a218ba294c7fc667" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35003 } }
I20260807 19:43:20.740324 10955 raft_consensus.cc:374] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:20.740458 10955 raft_consensus.cc:729] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b33cacd844b4ec4beca1ca664f289af, State: Initialized, Role: FOLLOWER
I20260807 19:43:20.740832 10782 heartbeater.cc:503] Master 127.30.208.62:45231 was elected leader, sending a full tablet report...
I20260807 19:43:20.741344 10955 consensus_queue.cc:260] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e7fb61ca4ba4cc7accc63645a66c117" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 43241 } } peers { permanent_uuid: "1b33cacd844b4ec4beca1ca664f289af" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35727 } } peers { permanent_uuid: "21eafb859c1d4847a218ba294c7fc667" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35003 } }
I20260807 19:43:20.741791 10957 ts_tablet_manager.cc:1428] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260807 19:43:20.741554 10956 raft_consensus.cc:348] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e7fb61ca4ba4cc7accc63645a66c117" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 43241 } } peers { permanent_uuid: "1b33cacd844b4ec4beca1ca664f289af" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35727 } } peers { permanent_uuid: "21eafb859c1d4847a218ba294c7fc667" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35003 } }
I20260807 19:43:20.742055 10956 raft_consensus.cc:374] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:43:20.742172 10956 raft_consensus.cc:729] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21eafb859c1d4847a218ba294c7fc667, State: Initialized, Role: FOLLOWER
I20260807 19:43:20.742887 10956 consensus_queue.cc:260] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e7fb61ca4ba4cc7accc63645a66c117" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 43241 } } peers { permanent_uuid: "1b33cacd844b4ec4beca1ca664f289af" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35727 } } peers { permanent_uuid: "21eafb859c1d4847a218ba294c7fc667" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35003 } }
I20260807 19:43:20.744069 10932 heartbeater.cc:503] Master 127.30.208.62:45231 was elected leader, sending a full tablet report...
I20260807 19:43:20.744719 10956 ts_tablet_manager.cc:1428] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260807 19:43:20.748943 10955 ts_tablet_manager.cc:1428] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af: Time spent starting tablet: real 0.020s user 0.019s sys 0.000s
I20260807 19:43:20.803748 10964 raft_consensus.cc:482] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:43:20.804081 10964 raft_consensus.cc:504] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e7fb61ca4ba4cc7accc63645a66c117" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 43241 } } peers { permanent_uuid: "1b33cacd844b4ec4beca1ca664f289af" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35727 } } peers { permanent_uuid: "21eafb859c1d4847a218ba294c7fc667" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35003 } }
I20260807 19:43:20.805950 10964 leader_election.cc:288] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e7fb61ca4ba4cc7accc63645a66c117 (127.30.208.2:43241), 21eafb859c1d4847a218ba294c7fc667 (127.30.208.3:35003)
W20260807 19:43:20.807161 10632 tablet.cc:2369] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:43:20.813349 10954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.806561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:55275 (local address 127.30.208.3:35003)
0807 19:43:20.806791 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.806797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.807310 (+ 513us) server_negotiation.cc:407] Connection header received
0807 19:43:20.807697 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.807702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.807789 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.807896 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:20.808595 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.809421 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.811442 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.812297 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.812436 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.812492 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.812893 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.813026 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:43:20.813431 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.806244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:35003 (local address 127.30.208.1:55275)
0807 19:43:20.807172 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:20.807196 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:20.807447 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:20.807927 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:20.807932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:20.807955 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:20.808437 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:20.808447 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.809552 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.809556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:20.811270 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:20.811284 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.812526 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.812533 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:20.812704 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.812745 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 19:43:20.812845 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 19:43:20.812959 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"mutex_wait_us":79,"spinlock_wait_cycles":21888,"thread_start_us":981,"threads_started":1}
I20260807 19:43:20.813524 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.806377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:60581 (local address 127.30.208.2:43241)
0807 19:43:20.806592 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.806598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.807634 (+ 1036us) server_negotiation.cc:407] Connection header received
0807 19:43:20.807888 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.807894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.807974 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.808059 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:43:20.808649 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.809415 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.811696 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.812594 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.812747 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.812834 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.813140 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.813291 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:43:20.813810 10966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.806379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:43241 (local address 127.30.208.1:60581)
0807 19:43:20.807545 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0807 19:43:20.807562 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:43:20.807725 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:43:20.808089 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:43:20.808092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:43:20.808126 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:43:20.808483 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:20.808491 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.809548 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.809553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:20.811547 (+ 1994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:43:20.811562 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.812799 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.812804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:43:20.812969 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.813001 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:43:20.813101 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 19:43:20.813253 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1052,"mutex_wait_us":47,"thread_start_us":434,"threads_started":1}
I20260807 19:43:20.814584 10883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "333f3ba8dbc44186b31fdda9c989fe2a" candidate_uuid: "1b33cacd844b4ec4beca1ca664f289af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21eafb859c1d4847a218ba294c7fc667" is_pre_election: true
I20260807 19:43:20.814669 10732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "333f3ba8dbc44186b31fdda9c989fe2a" candidate_uuid: "1b33cacd844b4ec4beca1ca664f289af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e7fb61ca4ba4cc7accc63645a66c117" is_pre_election: true
I20260807 19:43:20.815084 10883 raft_consensus.cc:2454] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b33cacd844b4ec4beca1ca664f289af in term 0.
I20260807 19:43:20.815120 10732 raft_consensus.cc:2454] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b33cacd844b4ec4beca1ca664f289af in term 0.
I20260807 19:43:20.815718 10510 leader_election.cc:302] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b33cacd844b4ec4beca1ca664f289af, 21eafb859c1d4847a218ba294c7fc667, 3e7fb61ca4ba4cc7accc63645a66c117; no voters:
I20260807 19:43:20.816119 10964 raft_consensus.cc:2781] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:43:20.816283 10964 raft_consensus.cc:482] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:43:20.816391 10964 raft_consensus.cc:3037] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:20.818601 10964 raft_consensus.cc:504] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e7fb61ca4ba4cc7accc63645a66c117" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 43241 } } peers { permanent_uuid: "1b33cacd844b4ec4beca1ca664f289af" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35727 } } peers { permanent_uuid: "21eafb859c1d4847a218ba294c7fc667" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35003 } }
I20260807 19:43:20.819322 10964 leader_election.cc:288] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [CANDIDATE]: Term 1 election: Requested vote from peers 3e7fb61ca4ba4cc7accc63645a66c117 (127.30.208.2:43241), 21eafb859c1d4847a218ba294c7fc667 (127.30.208.3:35003)
I20260807 19:43:20.819711 10732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "333f3ba8dbc44186b31fdda9c989fe2a" candidate_uuid: "1b33cacd844b4ec4beca1ca664f289af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e7fb61ca4ba4cc7accc63645a66c117"
I20260807 19:43:20.819877 10883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "333f3ba8dbc44186b31fdda9c989fe2a" candidate_uuid: "1b33cacd844b4ec4beca1ca664f289af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21eafb859c1d4847a218ba294c7fc667"
I20260807 19:43:20.819936 10732 raft_consensus.cc:3037] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:20.820078 10883 raft_consensus.cc:3037] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:43:20.823252 10883 raft_consensus.cc:2454] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b33cacd844b4ec4beca1ca664f289af in term 1.
I20260807 19:43:20.823252 10732 raft_consensus.cc:2454] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b33cacd844b4ec4beca1ca664f289af in term 1.
I20260807 19:43:20.823788 10509 leader_election.cc:302] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b33cacd844b4ec4beca1ca664f289af, 3e7fb61ca4ba4cc7accc63645a66c117; no voters:
I20260807 19:43:20.824126 10964 raft_consensus.cc:2781] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:43:20.824623 10964 raft_consensus.cc:686] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [term 1 LEADER]: Becoming Leader. State: Replica: 1b33cacd844b4ec4beca1ca664f289af, State: Running, Role: LEADER
I20260807 19:43:20.825107 10964 consensus_queue.cc:237] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e7fb61ca4ba4cc7accc63645a66c117" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 43241 } } peers { permanent_uuid: "1b33cacd844b4ec4beca1ca664f289af" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35727 } } peers { permanent_uuid: "21eafb859c1d4847a218ba294c7fc667" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35003 } }
I20260807 19:43:20.830044 10416 catalog_manager.cc:5495] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b33cacd844b4ec4beca1ca664f289af (127.30.208.1). New cstate: current_term: 1 leader_uuid: "1b33cacd844b4ec4beca1ca664f289af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e7fb61ca4ba4cc7accc63645a66c117" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 43241 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b33cacd844b4ec4beca1ca664f289af" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 35727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21eafb859c1d4847a218ba294c7fc667" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35003 } health_report { overall_health: UNKNOWN } } }
W20260807 19:43:20.838652 10783 tablet.cc:2369] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:43:20.856369 10933 tablet.cc:2369] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:43:20.863032 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.857596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57978 (local address 127.30.208.1:35727)
0807 19:43:20.857803 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.857809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.857829 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:43:20.858052 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.858056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.858149 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.858290 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:43:20.858760 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.859496 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.860859 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.861318 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.861425 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.861988 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:43:20.862498 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:43:20.862608 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.862731 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.862807 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 19:43:20.871927 10883 raft_consensus.cc:1261] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667 [term 1 FOLLOWER]: Refusing update from remote peer 1b33cacd844b4ec4beca1ca664f289af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:20.871927 10732 raft_consensus.cc:1261] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117 [term 1 FOLLOWER]: Refusing update from remote peer 1b33cacd844b4ec4beca1ca664f289af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:43:20.873099 10968 consensus_queue.cc:1035] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e7fb61ca4ba4cc7accc63645a66c117" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 43241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:20.873566 10964 consensus_queue.cc:1035] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [LEADER]: Connected to new peer: Peer: permanent_uuid: "21eafb859c1d4847a218ba294c7fc667" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 35003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:43:20.886165 10971 mvcc.cc:205] Tried to move back new op lower bound from 7315995856360423424 to 7315995856183746560. Current Snapshot: MvccSnapshot[applied={T|T < 7315995856360423424}]
I20260807 19:43:20.901387 10416 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:43:20.901684 10416 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:43:20.905303 10416 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 673952fd98fa4292ae661702432d5b8d: Sending DeleteTablet for 3 replicas of tablet 333f3ba8dbc44186b31fdda9c989fe2a
I20260807 19:43:20.906646 10863 tablet_service.cc:1514] Processing DeleteTablet for tablet 333f3ba8dbc44186b31fdda9c989fe2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:20 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:59158
I20260807 19:43:20.906701 10709 tablet_service.cc:1514] Processing DeleteTablet for tablet 333f3ba8dbc44186b31fdda9c989fe2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:20 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:49514
I20260807 19:43:20.907410 10558 tablet_service.cc:1514] Processing DeleteTablet for tablet 333f3ba8dbc44186b31fdda9c989fe2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:43:20 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:57962
I20260807 19:43:20.907611 10983 tablet_replica.cc:330] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117: stopping tablet replica
I20260807 19:43:20.908051 10983 raft_consensus.cc:2229] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:43:20.908586 10983 raft_consensus.cc:2258] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:20.908720 10984 tablet_replica.cc:330] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af: stopping tablet replica
I20260807 19:43:20.908820 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 10490
I20260807 19:43:20.909111 10984 raft_consensus.cc:2229] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:43:20.909391 10982 tablet_replica.cc:330] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667: stopping tablet replica
I20260807 19:43:20.909793 10982 raft_consensus.cc:2229] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:43:20.909837 10984 raft_consensus.cc:2258] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:20.910266 10982 raft_consensus.cc:2258] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:43:20.912035 10984 ts_tablet_manager.cc:1905] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:20.913399 10982 ts_tablet_manager.cc:1905] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:20.914033 10983 ts_tablet_manager.cc:1905] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:43:20.920390 10982 ts_tablet_manager.cc:1918] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:43:20.920677 10982 log.cc:1196] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/333f3ba8dbc44186b31fdda9c989fe2a
I20260807 19:43:20.920897 10984 ts_tablet_manager.cc:1918] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:43:20.921012 10983 ts_tablet_manager.cc:1918] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:43:20.921118 10984 log.cc:1196] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/333f3ba8dbc44186b31fdda9c989fe2a
I20260807 19:43:20.921276 10983 log.cc:1196] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/333f3ba8dbc44186b31fdda9c989fe2a
I20260807 19:43:20.921543 10982 ts_tablet_manager.cc:1939] T 333f3ba8dbc44186b31fdda9c989fe2a P 21eafb859c1d4847a218ba294c7fc667: Deleting consensus metadata
I20260807 19:43:20.921702 10984 ts_tablet_manager.cc:1939] T 333f3ba8dbc44186b31fdda9c989fe2a P 1b33cacd844b4ec4beca1ca664f289af: Deleting consensus metadata
I20260807 19:43:20.921864 10983 ts_tablet_manager.cc:1939] T 333f3ba8dbc44186b31fdda9c989fe2a P 3e7fb61ca4ba4cc7accc63645a66c117: Deleting consensus metadata
I20260807 19:43:20.923626 10403 catalog_manager.cc:4841] TS 1b33cacd844b4ec4beca1ca664f289af (127.30.208.1:35727): tablet 333f3ba8dbc44186b31fdda9c989fe2a (table test-table [id=e7aadb41146c400daa3f26bf68b3b439]) successfully deleted
I20260807 19:43:20.923787 10402 catalog_manager.cc:4841] TS 3e7fb61ca4ba4cc7accc63645a66c117 (127.30.208.2:43241): tablet 333f3ba8dbc44186b31fdda9c989fe2a (table test-table [id=e7aadb41146c400daa3f26bf68b3b439]) successfully deleted
I20260807 19:43:20.923982 10403 catalog_manager.cc:4841] TS 21eafb859c1d4847a218ba294c7fc667 (127.30.208.3:35003): tablet 333f3ba8dbc44186b31fdda9c989fe2a (table test-table [id=e7aadb41146c400daa3f26bf68b3b439]) successfully deleted
Aug 07 19:43:20 dist-test-slave-vq2q krb5kdc[10366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:43:20.943069 10952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:20.912111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57994 (local address 127.30.208.1:35727)
0807 19:43:20.912356 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:20.912362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:20.923696 (+ 11334us) server_negotiation.cc:407] Connection header received
0807 19:43:20.924028 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:20.924034 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:20.924140 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:20.924233 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:20.926220 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.927412 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.929259 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:20.930005 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:20.930138 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:20.935504 (+ 5366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:20.935524 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:20.935545 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:20.935599 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:20.939712 (+ 4113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:20.940642 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:20.940653 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:20.940657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:20.940854 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:20.941310 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:20.941315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:20.941322 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:20.941824 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:20.942146 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:20.942430 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 19:43:20.942688 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 19:43:20.943951 10626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:21.045032 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 10490
I20260807 19:43:21.068150 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 10639
Aug 07 19:43:21 dist-test-slave-vq2q krb5kdc[10366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:43:21.086357 10950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:21.069347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49530 (local address 127.30.208.2:43241)
0807 19:43:21.069538 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:21.069544 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:21.069565 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:43:21.069636 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:21.069641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:21.069708 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:21.069806 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:21.071062 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:21.072239 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:21.073973 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:21.074633 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:21.074770 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:21.079168 (+ 4398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:21.079189 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:21.079210 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:21.079260 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:21.083216 (+ 3956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:21.083909 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:21.083919 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:21.083923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:21.084087 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:21.084430 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:21.084435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:21.084438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:21.085037 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:21.085406 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:21.085728 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 19:43:21.085965 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:43:21.087246 10777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:21.175498 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 10639
I20260807 19:43:21.192790 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 10788
Aug 07 19:43:21 dist-test-slave-vq2q krb5kdc[10366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:43:21.207470 10954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:43:21.193539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59172 (local address 127.30.208.3:35003)
0807 19:43:21.193741 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:43:21.193747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:43:21.193768 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:43:21.193943 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:43:21.193948 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:43:21.194024 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:43:21.194107 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:43:21.195390 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:21.196100 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:21.197301 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:43:21.197698 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:43:21.197808 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:43:21.202030 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:43:21.202046 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:43:21.202063 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:43:21.202098 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:43:21.204778 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:21.205574 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:21.205581 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:21.205583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:21.205774 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:43:21.206073 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:43:21.206076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:43:21.206078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:43:21.206486 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:43:21.206744 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 19:43:21.206987 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 19:43:21.207159 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:43:21.208096 10926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:21.296981 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 10788
I20260807 19:43:21.315562 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 10379
I20260807 19:43:21.316727 10453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:43:21.426030 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 10379
2026-08-07T19:43:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4327 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:43:21 dist-test-slave-vq2q krb5kdc[10993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:43:21 dist-test-slave-vq2q krb5kdc[10993](info): set up 2 sockets
Aug 07 19:43:21 dist-test-slave-vq2q krb5kdc[10993](info): commencing operation
krb5kdc: starting...
W20260807 19:43:22.508090 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:43:22 dist-test-slave-vq2q krb5kdc[10993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:43:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:43:22Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/rangeradmin_127.30.208.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/rangeradmin_127.30.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/rangerlookup_127.30.208.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/rangerlookup_127.30.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/HTTP_127.30.208.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/HTTP_127.30.208.20@KRBTEST.COM.keytab.
I20260807 19:43:22.892908 31552 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXLhxOI/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/postgres -l logfile start
2026-08-07 19:43:25.511 UTC [11025] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 19:43:25.511 UTC [11025] LOG: listening on IPv4 address "127.30.208.20", port 44549
2026-08-07 19:43:25.515 UTC [11025] LOG: listening on Unix socket "/tmp/.s.PGSQL.44549"
2026-08-07 19:43:25.538 UTC [11028] LOG: database system was shut down at 2026-08-07 19:43:23 UTC
2026-08-07 19:43:25.542 UTC [11025] LOG: database system is ready to accept connections
I20260807 19:43:26.495963 31552 mini_postgres.cc:96] Postgres bound to 44549
2026-08-07 19:43:26.518 UTC [11036] FATAL: database "slave" does not exist
127.30.208.20:44549 - accepting connections
I20260807 19:43:26.519412 31552 mini_ranger.cc:161] Starting Ranger...
I20260807 19:43:26.550649 31552 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 19:43:27.085458 31552 mini_ranger.cc:87] Created ranger Postgres database
I20260807 19:43:27.085630 31552 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-admin
2026-08-07 19:43:27,401 [I] DB FLAVOR :POSTGRES
2026-08-07 19:43:27,401 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:43:27,402 [I] Checking connection..
2026-08-07 19:43:27,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 19:43:27,654 [I] Checking connection passed.
2026-08-07 19:43:27,654 [I] --------- Verifying version history table ---------
2026-08-07 19:43:27,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:43:27,931 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:43:27,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:43:28,179 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:43:28,179 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 19:43:28,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 19:43:28.433 UTC [11107] WARNING: there is no transaction in progress
2026-08-07 19:43:28,440 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 19:43:28,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:43:28,677 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 19:43:28,678 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:43:28,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 19:43:28,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 19:43:29,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vq2q.c.gcp-upstream.internal','N') ;"
2026-08-07 19:43:29,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:43:29,719 [I] Table x_portal_user does not exist in database ranger
2026-08-07 19:43:29,720 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:43:29,985 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 19:43:29,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:43:30,248 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:43:30,510 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 19:43:30,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 19:43:31.504 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.520 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.537 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.554 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.570 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.587 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.643 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.650 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.665 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.675 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.686 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.700 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.710 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.717 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.725 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:31.732 UTC [11227] WARNING: there is no transaction in progress
2026-08-07 19:43:32,441 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 19:43:32,441 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 19:43:32,689 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 19:43:32,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:43:32,948 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 19:43:32,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:43:33,220 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 19:43:33,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:43:33,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:43:33,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vq2q.c.gcp-upstream.internal';"
2026-08-07 19:43:33,991 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 19:43:33,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 19:43:34,245 [I] DB_PATCHES have already been applied
I20260807 19:43:34.251925 31552 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXLhxOI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 19:43:34.252048 31552 mini_ranger.cc:193] Using host: 127.30.208.20
I20260807 19:43:34.271714 31552 mini_ranger.cc:227] Ranger admin URL: http://127.30.208.20:46341
Aug 07, 2026 7:43:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:43:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:43:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 7:43:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 7:43:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 7:43:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 7:43:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 19:43:35.347332 31552 mini_ranger.cc:160] Time spent starting Ranger: real 8.828s user 0.002s sys 0.059s
Aug 07, 2026 7:43:55 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 7:43:55 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 7:43:55 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 7:43:55 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 7:43:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 7:43:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 7:43:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 7:43:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 7:43:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 7:43:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 7:43:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 7:43:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 7:43:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 7:43:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 7:43:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 7:43:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 7:43:55 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 7:43:56 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 19:43:57.382515 31552 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/rangerkms_127.30.208.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/rangerkms_127.30.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.208.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.208.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.208.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/HTTP_127.30.208.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.208.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/HTTP_127.30.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 19:43:57 dist-test-slave-vq2q krb5kdc[10993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131837, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 19:43:57.579479 31552 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 19:43:57.604355 31552 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 19:43:58.502909 31552 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 19:43:58.503058 31552 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-kms
I20260807 19:43:58.503227 31552 mini_ranger_kms.cc:227] Using postgres at 127.30.208.20:44549
2026-08-07 19:43:59,313 [I] DB FLAVOR :POSTGRES
2026-08-07 19:43:59,314 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:43:59,314 [I] Checking connection
2026-08-07 19:43:59,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 19:43:59,563 [I] connection success
2026-08-07 19:43:59,563 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 19:43:59,563 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 19:43:59,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 19:43:59,802 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 19:43:59,802 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:43:59,802 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 19:43:59,802 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:44549/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 19:44:00,069 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 19:44:00.542307 31552 mini_ranger_kms.cc:310] Created kms service
I20260807 19:44:00.730890 31552 mini_ranger_kms.cc:326] Created kudu user
I20260807 19:44:00.778141 31552 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 19:44:00 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131837, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:44:01.209307 31552 mini_ranger_kms.cc:384] Added ranger policy
I20260807 19:44:01.210819 31552 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 19:44:01.210891 31552 mini_ranger_kms.cc:240] Using host: 127.30.208.20
I20260807 19:44:01.236032 31552 mini_ranger_kms.cc:276] Ranger KMS PID: 11446
I20260807 19:44:01.236233 31552 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.208.20:56651
Aug 07, 2026 7:44:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:44:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:44:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 7:44:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 7:44:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 7:44:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 19:44:03 dist-test-slave-vq2q krb5kdc[10993](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131843, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.208.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:03.526562 31552 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.947s user 0.143s sys 0.445s
I20260807 19:44:03.526929 31552 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 19:44:03.527173 31552 mini_ranger_kms.cc:401] 127.30.208.20:56651/kms/v1/keys
Aug 07 19:44:03 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131843, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.208.20@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
Aug 07 19:44:04 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131843, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.208.20@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
Aug 07 19:44:06 dist-test-slave-vq2q krb5kdc[10993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:44:06.869773 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:34545
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:34545
--ranger_config_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.208.20:56651/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:07.334527 11498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:07.334960 11498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:07.335197 11498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:07.343961 11498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:07.344138 11498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:07.344229 11498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:07.344316 11498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:07.344391 11498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:07.356484 11498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.208.20:56651/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:34545
--ranger_config_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:34545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:44:07.357954 11498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:07.359490 11498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:07.374277 11507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:07.374214 11503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:07.374521 11505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:07.375641 11498 server_base.cc:1034] running on GCE node
I20260807 19:44:07.376571 11498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:07.378214 11498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:07.379443 11498 hybrid_clock.cc:648] HybridClock initialized: now 1786131847379387 us; error 36 us; skew 500 ppm
I20260807 19:44:07.379809 11498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:07 dist-test-slave-vq2q krb5kdc[10993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131847, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:07.387916 11498 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:44:07.389711 11498 webserver.cc:460] Webserver started at http://127.30.208.62:44417/ using document root <none> and password file <none>
I20260807 19:44:07.390489 11498 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:07.390614 11498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:07.390941 11498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:44:07 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131847, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:44:07.452523 11498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "354c9d305b4c4bc2a636adc3f15f27d0"
format_stamp: "Formatted at 2026-08-07 19:44:07 on dist-test-slave-vq2q"
server_key: "6b791dfc606af06adaf2618dc1ffc6f4"
server_key_iv: "2f8ac42c049e5b7ba28d8453a9289e0a"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:07.453470 11498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "354c9d305b4c4bc2a636adc3f15f27d0"
format_stamp: "Formatted at 2026-08-07 19:44:07 on dist-test-slave-vq2q"
server_key: "6b791dfc606af06adaf2618dc1ffc6f4"
server_key_iv: "2f8ac42c049e5b7ba28d8453a9289e0a"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:07.460532 11498 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260807 19:44:07.478755 11515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:07.480548 11498 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260807 19:44:07.480754 11498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "354c9d305b4c4bc2a636adc3f15f27d0"
format_stamp: "Formatted at 2026-08-07 19:44:07 on dist-test-slave-vq2q"
server_key: "6b791dfc606af06adaf2618dc1ffc6f4"
server_key_iv: "2f8ac42c049e5b7ba28d8453a9289e0a"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:07.481109 11498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:07.517163 11498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:07.521771 11498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:07.522140 11498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:07.549985 11498 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:34545
I20260807 19:44:07.550081 11573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:34545 every 8 connection(s)
I20260807 19:44:07.551411 11498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:07.555745 11574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:07.559370 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 11498
I20260807 19:44:07.559592 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:44:07.560397 31552 external_mini_cluster.cc:1428] Setting key 415337d64a40da40f0d84ba7ebd5ecde
I20260807 19:44:07.570062 11574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0: Bootstrap starting.
I20260807 19:44:07.574019 11574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0: Neither blocks nor log segments found. Creating new log.
Aug 07 19:44:07 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:44:07.575251 11574 log.cc:824] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:07.578433 11574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0: No bootstrap required, opened a new log
I20260807 19:44:07.583623 11577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:07.562722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34882 (local address 127.30.208.62:34545)
0807 19:44:07.563839 (+ 1117us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:07.563850 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:07.563894 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:44:07.565715 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:07.565738 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:07.566135 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:07.566563 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:07.567911 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:07.569459 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:07.570780 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:07.571649 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:07.571831 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:07.575724 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:07.575733 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:07.575747 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:07.575795 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:07.579210 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:07.580034 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:07.580041 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:07.580047 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:07.580216 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:07.580654 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:07.580659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:07.580663 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:07.581289 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:07.581598 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:07.582061 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 19:44:07.582489 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":858,"thread_start_us":617,"threads_started":1}
I20260807 19:44:07.588443 11574 raft_consensus.cc:348] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354c9d305b4c4bc2a636adc3f15f27d0" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34545 } }
I20260807 19:44:07.588850 11574 raft_consensus.cc:374] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:07.588975 11574 raft_consensus.cc:729] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 354c9d305b4c4bc2a636adc3f15f27d0, State: Initialized, Role: FOLLOWER
I20260807 19:44:07.589561 11574 consensus_queue.cc:260] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354c9d305b4c4bc2a636adc3f15f27d0" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34545 } }
I20260807 19:44:07.589774 11574 raft_consensus.cc:388] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:07.589934 11574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:07.590142 11574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:07.592595 11574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354c9d305b4c4bc2a636adc3f15f27d0" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34545 } }
I20260807 19:44:07.593143 11574 leader_election.cc:302] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 354c9d305b4c4bc2a636adc3f15f27d0; no voters:
I20260807 19:44:07.593950 11574 leader_election.cc:288] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:07.594075 11579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:07.594971 11579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [term 1 LEADER]: Becoming Leader. State: Replica: 354c9d305b4c4bc2a636adc3f15f27d0, State: Running, Role: LEADER
I20260807 19:44:07.595623 11579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354c9d305b4c4bc2a636adc3f15f27d0" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34545 } }
I20260807 19:44:07.596117 11574 sys_catalog.cc:564] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:07.598917 11580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "354c9d305b4c4bc2a636adc3f15f27d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354c9d305b4c4bc2a636adc3f15f27d0" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34545 } } }
I20260807 19:44:07.599960 11580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:07.599050 11581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 354c9d305b4c4bc2a636adc3f15f27d0. Latest consensus state: current_term: 1 leader_uuid: "354c9d305b4c4bc2a636adc3f15f27d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354c9d305b4c4bc2a636adc3f15f27d0" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34545 } } }
I20260807 19:44:07.600507 11581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0 [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:07.606592 11588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:07.613440 11588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:07.616132 11574 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 19:44:07.635097 11588 catalog_manager.cc:1347] Generated new cluster ID: e7fb10e57ce0405cb959aec71d6acde3
I20260807 19:44:07.635251 11588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:07.651767 11588 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:07.653436 11588 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:07.665004 11588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0: Generated new TSK 0
I20260807 19:44:07.666038 11588 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:44:08.535284 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.208.20:56651/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34545
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:44:08 dist-test-slave-vq2q krb5kdc[10993](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131848, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 19:44:09.071520 11626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:09.072104 11626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:09.072410 11626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:09.081332 11626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:09.081521 11626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:09.081722 11626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:44:09.095510 11626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.208.20:56651/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:44:09.096923 11626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:09.098399 11626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:09.118418 11637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:09.119524 11638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:09.127094 11640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:09.127416 11626 server_base.cc:1034] running on GCE node
I20260807 19:44:09.128259 11626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:09.129518 11626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:09.130875 11626 hybrid_clock.cc:648] HybridClock initialized: now 1786131849130772 us; error 85 us; skew 500 ppm
I20260807 19:44:09.131194 11626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:09 dist-test-slave-vq2q krb5kdc[10993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131849, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:09.134753 11626 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:44:09.142009 11626 webserver.cc:460] Webserver started at http://127.30.208.1:46387/ using document root <none> and password file <none>
I20260807 19:44:09.142755 11626 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:09.142885 11626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:09.143198 11626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:44:09 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131849, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:44:09.162564 11626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "811f13a5c50248a5a1f3cfbc9f975688"
format_stamp: "Formatted at 2026-08-07 19:44:09 on dist-test-slave-vq2q"
server_key: "9baf1896bb18fe197bc2057ef09adc8a"
server_key_iv: "d5b04ae32541b6ca34cdb53eeb06c59e"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:09.164849 11626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "811f13a5c50248a5a1f3cfbc9f975688"
format_stamp: "Formatted at 2026-08-07 19:44:09 on dist-test-slave-vq2q"
server_key: "9baf1896bb18fe197bc2057ef09adc8a"
server_key_iv: "d5b04ae32541b6ca34cdb53eeb06c59e"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:09.174013 11626 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.000s
I20260807 19:44:09.199980 11648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:09.201887 11626 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:09.202158 11626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "811f13a5c50248a5a1f3cfbc9f975688"
format_stamp: "Formatted at 2026-08-07 19:44:09 on dist-test-slave-vq2q"
server_key: "9baf1896bb18fe197bc2057ef09adc8a"
server_key_iv: "d5b04ae32541b6ca34cdb53eeb06c59e"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:09.202559 11626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:09.227978 11626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:09.234045 11626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:09.234493 11626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:09.236020 11626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:09.238138 11626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:09.238250 11626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:09.238443 11626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:09.238526 11626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:44:09 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131848, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:44:09.391528 11626 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:41665
I20260807 19:44:09.392081 11773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:41665 every 8 connection(s)
I20260807 19:44:09.393767 11626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:44:09.402130 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 11626
I20260807 19:44:09.402379 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:44:09.403195 31552 external_mini_cluster.cc:1428] Setting key b18532bc9132d43351e82f54dab0f6a0
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
Aug 07 19:44:09 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131849, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:44:09.472501 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:09.443117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:38993 (local address 127.30.208.62:34545)
0807 19:44:09.449120 (+ 6003us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:09.449126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:09.449148 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:44:09.449260 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:09.449265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:09.449375 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:09.449486 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:09.452048 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.453216 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.455767 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.456595 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.456714 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:09.461731 (+ 5017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:09.461740 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:09.461745 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:09.461790 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:09.466138 (+ 4348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:09.467642 (+ 1504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:09.467648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:09.467652 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:09.467837 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:09.468474 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:09.468479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:09.468483 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:09.468873 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:09.469017 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:09.471743 (+ 2726us) server_negotiation.cc:299] Negotiation successful
0807 19:44:09.472072 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5850,"thread_start_us":10102,"threads_started":1}
I20260807 19:44:09.475333 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:09.443716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34545 (local address 127.30.208.1:38993)
0807 19:44:09.445160 (+ 1444us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:09.445506 (+ 346us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:09.448352 (+ 2846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:09.449959 (+ 1607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:09.449972 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:09.450660 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:09.451802 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:09.451841 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.453472 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.453478 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:09.455570 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:09.455598 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:09.457095 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:09.457104 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:09.457461 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:09.458316 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:09.458351 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:09.461377 (+ 3026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:09.466415 (+ 5038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:09.466426 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:09.466446 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:09.467472 (+ 1026us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:09.467994 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:09.468000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:09.468003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:09.468324 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:09.469112 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:09.469120 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:09.469682 (+ 562us) client_negotiation.cc:769] Sending connection context
0807 19:44:09.473049 (+ 3367us) client_negotiation.cc:240] Negotiation successful
0807 19:44:09.473688 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1145,"thread_start_us":887,"threads_started":1}
I20260807 19:44:09.477751 11774 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34545
I20260807 19:44:09.478185 11774 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:09.479274 11774 heartbeater.cc:511] Master 127.30.208.62:34545 requested a full tablet report, sending...
I20260807 19:44:09.482012 11537 ts_manager.cc:194] Registered new tserver with Master: 811f13a5c50248a5a1f3cfbc9f975688 (127.30.208.1:41665)
I20260807 19:44:09.488135 11537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:38993
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:44:09.499066 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.208.20:56651/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34545
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:44:09 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131848, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
W20260807 19:44:10.145483 11780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:10.145807 11780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:10.145936 11780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:10.154120 11780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:10.154268 11780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:10.154448 11780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:44:10.166061 11780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.208.20:56651/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:44:10.167382 11780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:10.168861 11780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:10.187633 11796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:10.187963 11795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:10.188755 11799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:10.190222 11780 server_base.cc:1034] running on GCE node
I20260807 19:44:10.191020 11780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:10.192147 11780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:10.193362 11780 hybrid_clock.cc:648] HybridClock initialized: now 1786131850193277 us; error 78 us; skew 500 ppm
I20260807 19:44:10.193678 11780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:10 dist-test-slave-vq2q krb5kdc[10993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131850, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:10.197556 11780 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:44:10.199602 11780 webserver.cc:460] Webserver started at http://127.30.208.2:39013/ using document root <none> and password file <none>
I20260807 19:44:10.200441 11780 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:10.200577 11780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:10.200907 11780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:44:10 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131850, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:44:10.219266 11780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "59cb080db55b4d4bb652b258b0037e1b"
format_stamp: "Formatted at 2026-08-07 19:44:10 on dist-test-slave-vq2q"
server_key: "cfb0a30c275618bb524376bc036e2e40"
server_key_iv: "1efabde0a0b50c04dd998a3e581cef8c"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:10.220527 11780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "59cb080db55b4d4bb652b258b0037e1b"
format_stamp: "Formatted at 2026-08-07 19:44:10 on dist-test-slave-vq2q"
server_key: "cfb0a30c275618bb524376bc036e2e40"
server_key_iv: "1efabde0a0b50c04dd998a3e581cef8c"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:10.228056 11780 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.003s
I20260807 19:44:10.248054 11809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:10.250239 11780 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.005s sys 0.001s
I20260807 19:44:10.250468 11780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "59cb080db55b4d4bb652b258b0037e1b"
format_stamp: "Formatted at 2026-08-07 19:44:10 on dist-test-slave-vq2q"
server_key: "cfb0a30c275618bb524376bc036e2e40"
server_key_iv: "1efabde0a0b50c04dd998a3e581cef8c"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:10.250782 11780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:10.288777 11780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:10.293063 11780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:10.293485 11780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:10.294961 11780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:10.296703 11780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:10.296779 11780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:10.296932 11780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:10.296990 11780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:10.358239 11780 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:44733
I20260807 19:44:10.358410 11927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:44733 every 8 connection(s)
I20260807 19:44:10.359714 11780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:44:10.364903 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 11780
I20260807 19:44:10.365082 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:44:10.365804 31552 external_mini_cluster.cc:1428] Setting key e59a89260d7c329178695c962944046a
Aug 07 19:44:10 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131850, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:44:10.393656 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:10.362750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:34825 (local address 127.30.208.62:34545)
0807 19:44:10.363261 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:10.363267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:10.364757 (+ 1490us) server_negotiation.cc:407] Connection header received
0807 19:44:10.367011 (+ 2254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:10.367017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:10.367155 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:10.367252 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:10.373335 (+ 6083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:10.374465 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:10.377445 (+ 2980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:10.378051 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:10.378189 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:10.382872 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:10.382878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:10.382881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:10.382933 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:10.386789 (+ 3856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:10.390335 (+ 3546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:10.390342 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:10.390346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:10.390526 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:10.391150 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:10.391155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:10.391159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:10.391603 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:10.391755 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:10.392910 (+ 1155us) server_negotiation.cc:299] Negotiation successful
0807 19:44:10.393166 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":380,"thread_start_us":244,"threads_started":1}
I20260807 19:44:10.395912 11931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:10.363217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34545 (local address 127.30.208.2:34825)
0807 19:44:10.364323 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:10.364376 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:10.366748 (+ 2372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:10.367568 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:10.367576 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:10.368283 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:10.369125 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:10.369152 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:10.374649 (+ 5497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:10.374655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:10.377137 (+ 2482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:10.377224 (+ 87us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:10.378331 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:10.378339 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:10.378710 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:10.379596 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:10.379622 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:10.382527 (+ 2905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:10.389402 (+ 6875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:10.389416 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:10.389435 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:10.390146 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:10.390695 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:10.390700 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:10.390704 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:10.391003 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:10.391898 (+ 895us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:10.391906 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:10.392324 (+ 418us) client_negotiation.cc:769] Sending connection context
0807 19:44:10.394046 (+ 1722us) client_negotiation.cc:240] Negotiation successful
0807 19:44:10.394662 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"thread_start_us":484,"threads_started":1}
I20260807 19:44:10.397612 11928 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34545
I20260807 19:44:10.397989 11928 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:10.398834 11928 heartbeater.cc:511] Master 127.30.208.62:34545 requested a full tablet report, sending...
I20260807 19:44:10.400447 11537 ts_manager.cc:194] Registered new tserver with Master: 59cb080db55b4d4bb652b258b0037e1b (127.30.208.2:44733)
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
I20260807 19:44:10.401481 11537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:34825
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:44:10.441517 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.208.20:56651/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34545
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:44:10.493265 11774 heartbeater.cc:503] Master 127.30.208.62:34545 was elected leader, sending a full tablet report...
W20260807 19:44:10.888893 11935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:10.889254 11935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:10.889396 11935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:10.897533 11935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:10.897663 11935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:10.897851 11935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:44:10.908985 11935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.208.20:56651/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:44:10.910386 11935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:10.911818 11935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:10.926867 11941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:10.926872 11942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:10.927640 11945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:10.928046 11935 server_base.cc:1034] running on GCE node
I20260807 19:44:10.928743 11935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:10.930004 11935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:10.931284 11935 hybrid_clock.cc:648] HybridClock initialized: now 1786131850931216 us; error 51 us; skew 500 ppm
I20260807 19:44:10.931627 11935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:10 dist-test-slave-vq2q krb5kdc[10993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131850, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:10.935189 11935 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:44:10.937100 11935 webserver.cc:460] Webserver started at http://127.30.208.3:36455/ using document root <none> and password file <none>
I20260807 19:44:10.937879 11935 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:10.937997 11935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:10.938314 11935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:44:10 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131850, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:44:10.954308 11935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "6f778b4ff7ca41edb7dce41bc3122f94"
format_stamp: "Formatted at 2026-08-07 19:44:10 on dist-test-slave-vq2q"
server_key: "34f2a0c81f1930389e7d1fe04013ba58"
server_key_iv: "eadd8679ede50bbe190f057562b929cc"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:10.955196 11935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "6f778b4ff7ca41edb7dce41bc3122f94"
format_stamp: "Formatted at 2026-08-07 19:44:10 on dist-test-slave-vq2q"
server_key: "34f2a0c81f1930389e7d1fe04013ba58"
server_key_iv: "eadd8679ede50bbe190f057562b929cc"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:10.961534 11935 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 19:44:10.978022 11955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:10.980022 11935 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:44:10.980228 11935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "6f778b4ff7ca41edb7dce41bc3122f94"
format_stamp: "Formatted at 2026-08-07 19:44:10 on dist-test-slave-vq2q"
server_key: "34f2a0c81f1930389e7d1fe04013ba58"
server_key_iv: "eadd8679ede50bbe190f057562b929cc"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:10.980594 11935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:11.017087 11935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:11.021792 11935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:11.022118 11935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:11.023165 11935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:11.025115 11935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:11.025223 11935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 19:44:11.025337 11935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:11.025403 11935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:11.075726 11935 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:33517
I20260807 19:44:11.075822 12079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:33517 every 8 connection(s)
I20260807 19:44:11.077169 11935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:44:11.084848 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 11935
I20260807 19:44:11.085018 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:44:11.085587 31552 external_mini_cluster.cc:1428] Setting key 1ed88ae235331a12b45735ca6a399072
Aug 07 19:44:11 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131850, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:44:11.101925 12082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.080382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:43113 (local address 127.30.208.62:34545)
0807 19:44:11.081028 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:11.081034 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:11.082144 (+ 1110us) server_negotiation.cc:407] Connection header received
0807 19:44:11.084255 (+ 2111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:11.084265 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:11.084342 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:11.084456 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:11.086347 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.087452 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.089307 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.090051 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.090179 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.094350 (+ 4171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:11.094355 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:11.094359 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:11.094391 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:11.097835 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:11.098796 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:11.098801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:11.098803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:11.099041 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:11.099658 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:11.099662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:11.099664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:11.100104 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:11.100247 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:11.101127 (+ 880us) server_negotiation.cc:299] Negotiation successful
0807 19:44:11.101392 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":252,"threads_started":1}
I20260807 19:44:11.103073 12083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.080848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34545 (local address 127.30.208.3:43113)
0807 19:44:11.081978 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:11.082023 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:11.083956 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:11.084704 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:11.084715 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:11.085420 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:11.086149 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:11.086168 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.087683 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.087689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:11.089135 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:11.089156 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.090369 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.090379 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:11.090704 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.091376 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:11.091399 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:11.094110 (+ 2711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:11.098013 (+ 3903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:11.098020 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:11.098034 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:11.098641 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:11.099176 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:11.099181 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:11.099184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:11.099512 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:11.100234 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:11.100242 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:11.100712 (+ 470us) client_negotiation.cc:769] Sending connection context
0807 19:44:11.101003 (+ 291us) client_negotiation.cc:240] Negotiation successful
0807 19:44:11.101681 (+ 678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":883,"thread_start_us":582,"threads_started":1}
I20260807 19:44:11.104784 12080 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34545
I20260807 19:44:11.105103 12080 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:11.106315 12080 heartbeater.cc:511] Master 127.30.208.62:34545 requested a full tablet report, sending...
I20260807 19:44:11.108047 11537 ts_manager.cc:194] Registered new tserver with Master: 6f778b4ff7ca41edb7dce41bc3122f94 (127.30.208.3:33517)
I20260807 19:44:11.109164 11537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:43113
I20260807 19:44:11.114286 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:44:11.135560 12082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.119646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38644 (local address 127.30.208.62:34545)
0807 19:44:11.119860 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:11.119867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:11.120729 (+ 862us) server_negotiation.cc:407] Connection header received
0807 19:44:11.120873 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:11.120877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:11.120957 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:11.121042 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:11.123003 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.123765 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.125673 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.126312 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.126410 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.129486 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:11.129492 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:11.129495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:11.129531 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:11.132798 (+ 3267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:11.133606 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:11.133610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:11.133613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:11.133737 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:11.134136 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:11.134139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:11.134141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:11.134481 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:11.134743 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:11.135076 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 19:44:11.135308 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:44:11.140600 11537 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:44:11.150532 11537 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:44:11.187779 12097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.165786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36498 (local address 127.30.208.3:33517)
0807 19:44:11.174496 (+ 8710us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:11.174521 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:11.175422 (+ 901us) server_negotiation.cc:407] Connection header received
0807 19:44:11.175748 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:11.175756 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:11.176000 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:11.176210 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:11.177916 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.179581 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.184005 (+ 4424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.186379 (+ 2374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.186534 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.186763 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:11.187076 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 19:44:11.187247 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8219,"thread_start_us":7831,"threads_started":1}
I20260807 19:44:11.187808 12100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.175911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50026 (local address 127.30.208.2:44733)
0807 19:44:11.177588 (+ 1677us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:11.177595 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:11.177618 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:44:11.177693 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:11.177697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:11.177916 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:11.178126 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:11.179075 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.181212 (+ 2137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.184120 (+ 2908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.185813 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.185999 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.186393 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:11.187033 (+ 640us) server_negotiation.cc:299] Negotiation successful
0807 19:44:11.187245 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1521,"thread_start_us":1418,"threads_started":1}
I20260807 19:44:11.187886 12096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.165653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:33517 (local address 127.0.0.1:36498)
0807 19:44:11.175306 (+ 9653us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:11.175326 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:11.175538 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:11.176577 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:11.176581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:11.176608 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:11.177708 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:11.177726 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.179695 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.179699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:11.183760 (+ 4061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:11.183791 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.186641 (+ 2850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.186647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:11.186867 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.186898 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 19:44:11.186987 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 19:44:11.187116 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9459,"mutex_wait_us":308,"thread_start_us":9203,"threads_started":1}
I20260807 19:44:11.188279 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.175874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:44733 (local address 127.0.0.1:50026)
0807 19:44:11.176863 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:11.176886 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:11.177028 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:11.178130 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:11.178135 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:11.178165 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:11.178588 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:11.178602 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.181358 (+ 2756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.181364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:11.183913 (+ 2549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:11.183928 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.186142 (+ 2214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.186156 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:11.186351 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.186403 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 19:44:11.186531 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 19:44:11.186770 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"mutex_wait_us":78,"thread_start_us":744,"threads_started":1}
I20260807 19:44:11.188721 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.166308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33222 (local address 127.30.208.1:41665)
0807 19:44:11.174540 (+ 8232us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:11.174548 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:11.174580 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:44:11.174691 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:11.174700 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:11.174992 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:11.175214 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:11.177547 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.179507 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.184104 (+ 4597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.186238 (+ 2134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.186395 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.186672 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:11.188001 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0807 19:44:11.188168 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8069,"thread_start_us":7905,"threads_started":1}
I20260807 19:44:11.189308 12095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.165568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:41665 (local address 127.0.0.1:33222)
0807 19:44:11.171598 (+ 6030us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:11.171638 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:11.171841 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:11.175580 (+ 3739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:11.175587 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:11.175633 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:11.177018 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:11.177031 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.179675 (+ 2644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.179680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:11.183882 (+ 4202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:11.183896 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.187288 (+ 3392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.187298 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:11.187546 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.187586 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 19:44:11.187688 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 19:44:11.187824 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5777,"mutex_wait_us":433,"thread_start_us":5598,"threads_started":1}
I20260807 19:44:11.192950 12007 tablet_service.cc:1467] Processing CreateTablet for tablet faa63d4b5d97404aa737068a6fbc6709 (DEFAULT_TABLE table=test-table [id=7080bc2d68414caf96b27715273bfd66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:11.192950 11862 tablet_service.cc:1467] Processing CreateTablet for tablet faa63d4b5d97404aa737068a6fbc6709 (DEFAULT_TABLE table=test-table [id=7080bc2d68414caf96b27715273bfd66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:11.193020 11703 tablet_service.cc:1467] Processing CreateTablet for tablet faa63d4b5d97404aa737068a6fbc6709 (DEFAULT_TABLE table=test-table [id=7080bc2d68414caf96b27715273bfd66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:44:11.194533 11703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faa63d4b5d97404aa737068a6fbc6709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:11.194586 11862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faa63d4b5d97404aa737068a6fbc6709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:11.194947 12007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet faa63d4b5d97404aa737068a6fbc6709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:11.208124 12101 tablet_bootstrap.cc:492] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688: Bootstrap starting.
I20260807 19:44:11.211086 12104 tablet_bootstrap.cc:492] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94: Bootstrap starting.
I20260807 19:44:11.214646 12101 tablet_bootstrap.cc:654] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:11.215409 12104 tablet_bootstrap.cc:654] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:11.215574 12102 tablet_bootstrap.cc:492] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b: Bootstrap starting.
I20260807 19:44:11.216722 12101 log.cc:824] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:11.216909 12104 log.cc:824] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:11.220043 12101 tablet_bootstrap.cc:492] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688: No bootstrap required, opened a new log
I20260807 19:44:11.220310 12101 ts_tablet_manager.cc:1397] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.001s
I20260807 19:44:11.220381 12102 tablet_bootstrap.cc:654] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:11.220511 12104 tablet_bootstrap.cc:492] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94: No bootstrap required, opened a new log
I20260807 19:44:11.220782 12104 ts_tablet_manager.cc:1397] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 19:44:11.221911 12102 log.cc:824] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:11.226308 12102 tablet_bootstrap.cc:492] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b: No bootstrap required, opened a new log
I20260807 19:44:11.226660 12102 ts_tablet_manager.cc:1397] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.002s
I20260807 19:44:11.230520 12101 raft_consensus.cc:348] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 33517 } } peers { permanent_uuid: "811f13a5c50248a5a1f3cfbc9f975688" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41665 } } peers { permanent_uuid: "59cb080db55b4d4bb652b258b0037e1b" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44733 } }
I20260807 19:44:11.230898 12101 raft_consensus.cc:374] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:11.231062 12101 raft_consensus.cc:729] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 811f13a5c50248a5a1f3cfbc9f975688, State: Initialized, Role: FOLLOWER
I20260807 19:44:11.231709 12101 consensus_queue.cc:260] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 33517 } } peers { permanent_uuid: "811f13a5c50248a5a1f3cfbc9f975688" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41665 } } peers { permanent_uuid: "59cb080db55b4d4bb652b258b0037e1b" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44733 } }
I20260807 19:44:11.234025 12101 ts_tablet_manager.cc:1428] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688: Time spent starting tablet: real 0.014s user 0.012s sys 0.003s
I20260807 19:44:11.233945 12104 raft_consensus.cc:348] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 33517 } } peers { permanent_uuid: "811f13a5c50248a5a1f3cfbc9f975688" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41665 } } peers { permanent_uuid: "59cb080db55b4d4bb652b258b0037e1b" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44733 } }
I20260807 19:44:11.234592 12104 raft_consensus.cc:374] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:11.234740 12104 raft_consensus.cc:729] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f778b4ff7ca41edb7dce41bc3122f94, State: Initialized, Role: FOLLOWER
I20260807 19:44:11.235383 12104 consensus_queue.cc:260] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 33517 } } peers { permanent_uuid: "811f13a5c50248a5a1f3cfbc9f975688" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41665 } } peers { permanent_uuid: "59cb080db55b4d4bb652b258b0037e1b" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44733 } }
I20260807 19:44:11.236810 12080 heartbeater.cc:503] Master 127.30.208.62:34545 was elected leader, sending a full tablet report...
I20260807 19:44:11.237459 12104 ts_tablet_manager.cc:1428] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94: Time spent starting tablet: real 0.017s user 0.005s sys 0.013s
I20260807 19:44:11.238188 12102 raft_consensus.cc:348] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 33517 } } peers { permanent_uuid: "811f13a5c50248a5a1f3cfbc9f975688" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41665 } } peers { permanent_uuid: "59cb080db55b4d4bb652b258b0037e1b" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44733 } }
I20260807 19:44:11.238631 12102 raft_consensus.cc:374] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:11.238755 12102 raft_consensus.cc:729] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59cb080db55b4d4bb652b258b0037e1b, State: Initialized, Role: FOLLOWER
I20260807 19:44:11.239332 12102 consensus_queue.cc:260] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 33517 } } peers { permanent_uuid: "811f13a5c50248a5a1f3cfbc9f975688" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41665 } } peers { permanent_uuid: "59cb080db55b4d4bb652b258b0037e1b" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44733 } }
I20260807 19:44:11.240717 11928 heartbeater.cc:503] Master 127.30.208.62:34545 was elected leader, sending a full tablet report...
I20260807 19:44:11.241561 12102 ts_tablet_manager.cc:1428] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b: Time spent starting tablet: real 0.015s user 0.008s sys 0.007s
I20260807 19:44:11.306160 12109 raft_consensus.cc:482] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:44:11.306510 12109 raft_consensus.cc:504] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 33517 } } peers { permanent_uuid: "811f13a5c50248a5a1f3cfbc9f975688" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41665 } } peers { permanent_uuid: "59cb080db55b4d4bb652b258b0037e1b" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44733 } }
I20260807 19:44:11.308772 12109 leader_election.cc:288] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 811f13a5c50248a5a1f3cfbc9f975688 (127.30.208.1:41665), 59cb080db55b4d4bb652b258b0037e1b (127.30.208.2:44733)
I20260807 19:44:11.316051 12083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.308937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:41665 (local address 127.30.208.3:53465)
0807 19:44:11.309230 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:11.309253 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:11.309393 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:11.309999 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:11.310003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:11.310034 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:11.310207 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:11.310218 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.311816 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.311821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:11.313363 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:11.313376 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.315232 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.315238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:11.315423 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.315460 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:44:11.315568 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 19:44:11.315717 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260807 19:44:11.316114 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.309056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:53465 (local address 127.30.208.1:41665)
0807 19:44:11.309279 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:11.309285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:11.309311 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:44:11.309584 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:11.309590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:11.309678 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:11.309775 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:11.310383 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.311655 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.313540 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.314881 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.315054 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.315137 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:11.315642 (+ 505us) server_negotiation.cc:299] Negotiation successful
0807 19:44:11.315826 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:44:11.316933 12100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.309610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:48489 (local address 127.30.208.2:44733)
0807 19:44:11.309811 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:11.309816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:11.310133 (+ 317us) server_negotiation.cc:407] Connection header received
0807 19:44:11.310286 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:11.310292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:11.310390 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:11.310476 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:44:11.311139 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.312056 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.314334 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.315661 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.315796 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.315866 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:11.316487 (+ 621us) server_negotiation.cc:299] Negotiation successful
0807 19:44:11.316615 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:44:11.317314 12111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.309163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:44733 (local address 127.30.208.3:48489)
0807 19:44:11.309988 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:11.310040 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:11.310158 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:11.310514 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:11.310519 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:11.310547 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:44:11.310965 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:11.310978 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.312242 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.312246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:11.314134 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:11.314149 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.315962 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.315971 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:11.316252 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.316334 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 19:44:11.316484 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 19:44:11.316677 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":473,"threads_started":1}
I20260807 19:44:11.317288 11724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faa63d4b5d97404aa737068a6fbc6709" candidate_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "811f13a5c50248a5a1f3cfbc9f975688" is_pre_election: true
I20260807 19:44:11.318054 11724 raft_consensus.cc:2454] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f778b4ff7ca41edb7dce41bc3122f94 in term 0.
I20260807 19:44:11.318549 11883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faa63d4b5d97404aa737068a6fbc6709" candidate_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59cb080db55b4d4bb652b258b0037e1b" is_pre_election: true
I20260807 19:44:11.318825 11956 leader_election.cc:302] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f778b4ff7ca41edb7dce41bc3122f94, 811f13a5c50248a5a1f3cfbc9f975688; no voters:
I20260807 19:44:11.319285 12109 raft_consensus.cc:2781] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:44:11.319154 11883 raft_consensus.cc:2454] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f778b4ff7ca41edb7dce41bc3122f94 in term 0.
I20260807 19:44:11.319542 12109 raft_consensus.cc:482] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:44:11.319691 12109 raft_consensus.cc:3037] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:11.322687 12109 raft_consensus.cc:504] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 33517 } } peers { permanent_uuid: "811f13a5c50248a5a1f3cfbc9f975688" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41665 } } peers { permanent_uuid: "59cb080db55b4d4bb652b258b0037e1b" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44733 } }
I20260807 19:44:11.323598 12109 leader_election.cc:288] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [CANDIDATE]: Term 1 election: Requested vote from peers 811f13a5c50248a5a1f3cfbc9f975688 (127.30.208.1:41665), 59cb080db55b4d4bb652b258b0037e1b (127.30.208.2:44733)
I20260807 19:44:11.324168 11883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faa63d4b5d97404aa737068a6fbc6709" candidate_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59cb080db55b4d4bb652b258b0037e1b"
I20260807 19:44:11.324345 11883 raft_consensus.cc:3037] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:11.326084 11724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "faa63d4b5d97404aa737068a6fbc6709" candidate_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "811f13a5c50248a5a1f3cfbc9f975688"
I20260807 19:44:11.326499 11724 raft_consensus.cc:3037] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:11.327121 11883 raft_consensus.cc:2454] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f778b4ff7ca41edb7dce41bc3122f94 in term 1.
I20260807 19:44:11.327696 11957 leader_election.cc:302] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59cb080db55b4d4bb652b258b0037e1b, 6f778b4ff7ca41edb7dce41bc3122f94; no voters:
I20260807 19:44:11.328210 12109 raft_consensus.cc:2781] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:11.328692 12109 raft_consensus.cc:686] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [term 1 LEADER]: Becoming Leader. State: Replica: 6f778b4ff7ca41edb7dce41bc3122f94, State: Running, Role: LEADER
W20260807 19:44:11.329350 12081 tablet.cc:2369] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:44:11.329325 12109 consensus_queue.cc:237] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 33517 } } peers { permanent_uuid: "811f13a5c50248a5a1f3cfbc9f975688" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41665 } } peers { permanent_uuid: "59cb080db55b4d4bb652b258b0037e1b" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44733 } }
I20260807 19:44:11.330006 11724 raft_consensus.cc:2454] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f778b4ff7ca41edb7dce41bc3122f94 in term 1.
I20260807 19:44:11.335580 11537 catalog_manager.cc:5495] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f778b4ff7ca41edb7dce41bc3122f94 (127.30.208.3). New cstate: current_term: 1 leader_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f778b4ff7ca41edb7dce41bc3122f94" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 33517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "811f13a5c50248a5a1f3cfbc9f975688" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59cb080db55b4d4bb652b258b0037e1b" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44733 } health_report { overall_health: UNKNOWN } } }
W20260807 19:44:11.363428 11929 tablet.cc:2369] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:44:11.384526 12097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.379216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36506 (local address 127.30.208.3:33517)
0807 19:44:11.379425 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:11.379430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:11.379449 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:44:11.379626 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:11.379630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:11.379751 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:11.379880 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:44:11.380268 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.381024 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.382506 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.382928 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.383024 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.383453 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:44:11.384017 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:44:11.384127 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:11.384188 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 19:44:11.384259 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:44:11.393682 11724 raft_consensus.cc:1261] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688 [term 1 FOLLOWER]: Refusing update from remote peer 6f778b4ff7ca41edb7dce41bc3122f94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:11.393757 11883 raft_consensus.cc:1261] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b [term 1 FOLLOWER]: Refusing update from remote peer 6f778b4ff7ca41edb7dce41bc3122f94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:44:11.394650 12109 consensus_queue.cc:1035] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59cb080db55b4d4bb652b258b0037e1b" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:11.395289 12113 consensus_queue.cc:1035] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "811f13a5c50248a5a1f3cfbc9f975688" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:44:11.405835 12116 mvcc.cc:205] Tried to move back new op lower bound from 7315996063297236992 to 7315996063051608064. Current Snapshot: MvccSnapshot[applied={T|T < 7315996063297236992}]
I20260807 19:44:11.408324 12118 mvcc.cc:205] Tried to move back new op lower bound from 7315996063297236992 to 7315996063051608064. Current Snapshot: MvccSnapshot[applied={T|T < 7315996063297236992}]
I20260807 19:44:11.419414 11537 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:11.419641 11537 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:44:11.423280 11537 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 354c9d305b4c4bc2a636adc3f15f27d0: Sending DeleteTablet for 3 replicas of tablet faa63d4b5d97404aa737068a6fbc6709
I20260807 19:44:11.424536 12007 tablet_service.cc:1514] Processing DeleteTablet for tablet faa63d4b5d97404aa737068a6fbc6709 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:11 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:36498
I20260807 19:44:11.424536 11703 tablet_service.cc:1514] Processing DeleteTablet for tablet faa63d4b5d97404aa737068a6fbc6709 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:11 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:33222
I20260807 19:44:11.424650 11862 tablet_service.cc:1514] Processing DeleteTablet for tablet faa63d4b5d97404aa737068a6fbc6709 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:44:11 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:50026
I20260807 19:44:11.427279 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 11626
I20260807 19:44:11.440330 12126 tablet_replica.cc:330] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688: stopping tablet replica
I20260807 19:44:11.440893 12126 raft_consensus.cc:2229] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:11.441496 12126 raft_consensus.cc:2258] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:11.442405 12128 tablet_replica.cc:330] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94: stopping tablet replica
I20260807 19:44:11.442860 12128 raft_consensus.cc:2229] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:44:11.443576 12128 raft_consensus.cc:2258] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:11.446431 12128 ts_tablet_manager.cc:1905] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:11.451174 12129 tablet_replica.cc:330] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b: stopping tablet replica
I20260807 19:44:11.451673 12129 raft_consensus.cc:2229] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:44:11.452222 12129 raft_consensus.cc:2258] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:44:11.452461 12126 ts_tablet_manager.cc:1905] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:44:11.454751 12128 ts_tablet_manager.cc:1918] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:11.455015 12128 log.cc:1196] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/faa63d4b5d97404aa737068a6fbc6709
I20260807 19:44:11.455618 12128 ts_tablet_manager.cc:1939] T faa63d4b5d97404aa737068a6fbc6709 P 6f778b4ff7ca41edb7dce41bc3122f94: Deleting consensus metadata
I20260807 19:44:11.457712 11519 catalog_manager.cc:4841] TS 6f778b4ff7ca41edb7dce41bc3122f94 (127.30.208.3:33517): tablet faa63d4b5d97404aa737068a6fbc6709 (table test-table [id=7080bc2d68414caf96b27715273bfd66]) successfully deleted
I20260807 19:44:11.458628 12129 ts_tablet_manager.cc:1905] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 19:44:11 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:44:11.459924 12126 ts_tablet_manager.cc:1918] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:11.460088 12126 log.cc:1196] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/faa63d4b5d97404aa737068a6fbc6709
I20260807 19:44:11.460611 12126 ts_tablet_manager.cc:1939] T faa63d4b5d97404aa737068a6fbc6709 P 811f13a5c50248a5a1f3cfbc9f975688: Deleting consensus metadata
I20260807 19:44:11.462513 11517 catalog_manager.cc:4841] TS 811f13a5c50248a5a1f3cfbc9f975688 (127.30.208.1:41665): tablet faa63d4b5d97404aa737068a6fbc6709 (table test-table [id=7080bc2d68414caf96b27715273bfd66]) successfully deleted
I20260807 19:44:11.466137 12129 ts_tablet_manager.cc:1918] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:44:11.466315 12129 log.cc:1196] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/faa63d4b5d97404aa737068a6fbc6709
I20260807 19:44:11.466786 12129 ts_tablet_manager.cc:1939] T faa63d4b5d97404aa737068a6fbc6709 P 59cb080db55b4d4bb652b258b0037e1b: Deleting consensus metadata
I20260807 19:44:11.468658 11519 catalog_manager.cc:4841] TS 59cb080db55b4d4bb652b258b0037e1b (127.30.208.2:44733): tablet faa63d4b5d97404aa737068a6fbc6709 (table test-table [id=7080bc2d68414caf96b27715273bfd66]) successfully deleted
I20260807 19:44:11.469291 12098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.429511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33226 (local address 127.30.208.1:41665)
0807 19:44:11.429712 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:11.429719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:11.444903 (+ 15184us) server_negotiation.cc:407] Connection header received
0807 19:44:11.445461 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:11.445467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:11.445563 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:11.445643 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:11.447595 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.448619 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.452670 (+ 4051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.453512 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.453629 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.461443 (+ 7814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:11.461461 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:11.461480 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:11.461526 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:11.465754 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:11.466789 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:11.466799 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:11.466803 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:11.466977 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:11.467484 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:11.467489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:11.467492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:11.467984 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:11.468301 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:11.468706 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 19:44:11.468944 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:44:11.469918 11769 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:11.622822 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 11626
I20260807 19:44:11.640801 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 11780
Aug 07 19:44:11 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:44:11.659070 12100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.641590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50034 (local address 127.30.208.2:44733)
0807 19:44:11.641752 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:11.641756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:11.641774 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:44:11.641977 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:11.641982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:11.642058 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:11.642131 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:11.643828 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.644864 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.646861 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.647455 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.647570 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.652993 (+ 5423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:11.653009 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:11.653026 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:11.653065 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:11.656045 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:11.656902 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:11.656912 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:11.656915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:11.657058 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:11.657491 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:11.657495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:11.657497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:11.657910 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:11.658157 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:11.658552 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 19:44:11.658730 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 19:44:11.659745 11923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:11.751989 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 11780
I20260807 19:44:11.776655 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 11935
Aug 07 19:44:11 dist-test-slave-vq2q krb5kdc[10993](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:44:11.797014 12097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:11.777505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36516 (local address 127.30.208.3:33517)
0807 19:44:11.778546 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:11.778553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:11.778571 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:44:11.778637 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:11.778642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:11.778728 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:11.778843 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:11.780646 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.781880 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.783226 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:11.783898 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:11.784037 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:11.789545 (+ 5508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:11.789565 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:11.789587 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:11.789646 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:11.793804 (+ 4158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:11.794571 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:11.794583 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:11.794586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:11.794810 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:11.795274 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:11.795279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:11.795283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:11.795748 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:11.796075 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:11.796346 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 19:44:11.796591 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307}
I20260807 19:44:11.797850 12075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:11.921736 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 11935
I20260807 19:44:11.941411 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 11498
I20260807 19:44:11.942713 11569 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:44:12.056861 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 11498
I20260807 19:44:12.079552 31552 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 19:44:12.415064 31552 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 19:44:12.415377 31552 mini_ranger.cc:66] Stopping Ranger...
I20260807 19:44:12.760996 31552 mini_ranger.cc:68] Stopped Ranger
2026-08-07 19:44:12.761 UTC [11025] LOG: received smart shutdown request
2026-08-07 19:44:12.766 UTC [11025] LOG: background worker "logical replication launcher" (PID 11034) exited with exit code 1
2026-08-07 19:44:12.766 UTC [11029] LOG: shutting down
2026-08-07 19:44:12.971 UTC [11025] LOG: database system is shut down
2026-08-07T19:44:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51652 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:44:13 dist-test-slave-vq2q krb5kdc[12146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:44:13 dist-test-slave-vq2q krb5kdc[12146](info): set up 2 sockets
Aug 07 19:44:13 dist-test-slave-vq2q krb5kdc[12146](info): commencing operation
krb5kdc: starting...
W20260807 19:44:14.173067 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.009s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:44:14 dist-test-slave-vq2q krb5kdc[12146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:44:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:44:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/rangeradmin_127.30.208.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/rangeradmin_127.30.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/rangerlookup_127.30.208.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/rangerlookup_127.30.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/HTTP_127.30.208.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/HTTP_127.30.208.20@KRBTEST.COM.keytab.
I20260807 19:44:14.546356 31552 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXLhxOI/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/postgres -l logfile start
2026-08-07 19:44:17.044 UTC [12178] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 19:44:17.044 UTC [12178] LOG: listening on IPv4 address "127.30.208.20", port 43451
2026-08-07 19:44:17.048 UTC [12178] LOG: listening on Unix socket "/tmp/.s.PGSQL.43451"
2026-08-07 19:44:17.070 UTC [12182] LOG: database system was shut down at 2026-08-07 19:44:15 UTC
2026-08-07 19:44:17.074 UTC [12178] LOG: database system is ready to accept connections
I20260807 19:44:18.023696 31552 mini_postgres.cc:96] Postgres bound to 43451
2026-08-07 19:44:18.045 UTC [12192] FATAL: database "slave" does not exist
127.30.208.20:43451 - accepting connections
I20260807 19:44:18.045730 31552 mini_ranger.cc:161] Starting Ranger...
I20260807 19:44:18.074785 31552 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 19:44:18.563731 31552 mini_ranger.cc:87] Created ranger Postgres database
I20260807 19:44:18.563890 31552 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-admin
2026-08-07 19:44:18,868 [I] DB FLAVOR :POSTGRES
2026-08-07 19:44:18,869 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:44:18,869 [I] Checking connection..
2026-08-07 19:44:18,869 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 19:44:19,117 [I] Checking connection passed.
2026-08-07 19:44:19,117 [I] --------- Verifying version history table ---------
2026-08-07 19:44:19,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:44:19,366 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:44:19,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:44:19,621 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 19:44:19,621 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 19:44:19,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 19:44:19.852 UTC [12266] WARNING: there is no transaction in progress
2026-08-07 19:44:19,860 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 19:44:19,860 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 19:44:20,096 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 19:44:20,097 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:44:20,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 19:44:20,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 19:44:20,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vq2q.c.gcp-upstream.internal','N') ;"
2026-08-07 19:44:20,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:44:21,085 [I] Table x_portal_user does not exist in database ranger
2026-08-07 19:44:21,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:44:21,334 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 19:44:21,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:44:21,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:44:21,836 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 19:44:21,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 19:44:22.790 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:22.805 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:22.821 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:22.836 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:22.851 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:22.868 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:22.923 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:22.930 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:22.944 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:22.954 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:22.964 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:22.977 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:22.987 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:22.993 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:23.000 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:23.006 UTC [12392] WARNING: there is no transaction in progress
2026-08-07 19:44:23,632 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 19:44:23,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 19:44:23,886 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 19:44:23,887 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 19:44:24,138 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 19:44:24,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 19:44:24,390 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 19:44:24,391 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 19:44:24,655 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 19:44:24,926 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vq2q.c.gcp-upstream.internal';"
2026-08-07 19:44:25,179 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 19:44:25,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 19:44:25,432 [I] DB_PATCHES have already been applied
I20260807 19:44:25.438297 31552 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXLhxOI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 19:44:25.438385 31552 mini_ranger.cc:193] Using host: 127.30.208.20
I20260807 19:44:25.456766 31552 mini_ranger.cc:227] Ranger admin URL: http://127.30.208.20:47365
Aug 07, 2026 7:44:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:44:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:44:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 7:44:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 7:44:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 7:44:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 7:44:26 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 19:44:26.549381 31552 mini_ranger.cc:160] Time spent starting Ranger: real 8.504s user 0.010s sys 0.046s
Aug 07, 2026 7:44:46 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 7:44:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 07, 2026 7:44:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 7:44:46 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 7:44:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 7:44:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 7:44:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 7:44:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 7:44:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 7:44:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 7:44:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 7:44:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 7:44:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 7:44:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 7:44:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 7:44:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 7:44:46 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 7:44:47 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 19:44:47.621870 31552 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/rangerkms_127.30.208.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/rangerkms_127.30.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.208.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.208.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.208.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/HTTP_127.30.208.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.208.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/HTTP_127.30.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 19:44:47 dist-test-slave-vq2q krb5kdc[12146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131887, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 19:44:47.827498 31552 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 19:44:47.851991 31552 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 19:44:48.746057 31552 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 19:44:48.746152 31552 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-kms
I20260807 19:44:48.746186 31552 mini_ranger_kms.cc:227] Using postgres at 127.30.208.20:43451
2026-08-07 19:44:49,128 [I] DB FLAVOR :POSTGRES
2026-08-07 19:44:49,128 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 19:44:49,128 [I] Checking connection
2026-08-07 19:44:49,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 19:44:49,384 [I] connection success
2026-08-07 19:44:49,384 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 19:44:49,384 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 19:44:49,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 19:44:49,647 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 19:44:49,647 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 19:44:49,647 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 19:44:49,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.208.20:43451/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 19:44:49,922 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 19:44:50.426102 31552 mini_ranger_kms.cc:310] Created kms service
I20260807 19:44:50.634690 31552 mini_ranger_kms.cc:326] Created kudu user
I20260807 19:44:50.684978 31552 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 19:44:50 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131887, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:44:51.121876 31552 mini_ranger_kms.cc:384] Added ranger policy
I20260807 19:44:51.122231 31552 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXLhxOI/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXLhxOI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 19:44:51.122291 31552 mini_ranger_kms.cc:240] Using host: 127.30.208.20
I20260807 19:44:51.141427 31552 mini_ranger_kms.cc:276] Ranger KMS PID: 12590
I20260807 19:44:51.141563 31552 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.208.20:34907
Aug 07, 2026 7:44:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 7:44:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 7:44:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 7:44:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 7:44:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 7:44:52 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 19:44:53.345918 31552 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.518s user 0.078s sys 0.336s
I20260807 19:44:53.346146 31552 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 19:44:53.346306 31552 mini_ranger_kms.cc:401] 127.30.208.20:34907/kms/v1/keys
Aug 07 19:44:53 dist-test-slave-vq2q krb5kdc[12146](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131893, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.208.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 19:44:53 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131893, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.208.20@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
Aug 07 19:44:54 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131893, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.208.20@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
Aug 07 19:44:56 dist-test-slave-vq2q krb5kdc[12146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:44:56.673949 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:44429
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:44429
--ranger_config_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.208.20:34907/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:44:57.013633 12645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:57.013937 12645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:57.014027 12645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:57.021596 12645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:44:57.021716 12645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:57.021768 12645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:57.021826 12645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:44:57.021876 12645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:44:57.032377 12645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.208.20:34907/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:44429
--ranger_config_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:44429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:44:57.033738 12645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:57.035023 12645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:57.046567 12653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:57.046624 12655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:57.046557 12652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:57.047197 12645 server_base.cc:1034] running on GCE node
I20260807 19:44:57.047979 12645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:57.049528 12645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:57.050729 12645 hybrid_clock.cc:648] HybridClock initialized: now 1786131897050689 us; error 37 us; skew 500 ppm
I20260807 19:44:57.051060 12645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:57 dist-test-slave-vq2q krb5kdc[12146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131897, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:57.057348 12645 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:44:57.058864 12645 webserver.cc:460] Webserver started at http://127.30.208.62:43669/ using document root <none> and password file <none>
I20260807 19:44:57.059587 12645 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:57.059705 12645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:57.060004 12645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:44:57 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131897, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:44:57.119932 12645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "dffcbb35cc764339a85d8fe133c2142f"
format_stamp: "Formatted at 2026-08-07 19:44:57 on dist-test-slave-vq2q"
server_key: "1c8aaa6246a902139858732d543f44b1"
server_key_iv: "427480bb5d080b368637000b27096ba6"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:57.121088 12645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "dffcbb35cc764339a85d8fe133c2142f"
format_stamp: "Formatted at 2026-08-07 19:44:57 on dist-test-slave-vq2q"
server_key: "1c8aaa6246a902139858732d543f44b1"
server_key_iv: "427480bb5d080b368637000b27096ba6"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:57.128237 12645 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260807 19:44:57.149154 12662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:57.151165 12645 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:44:57.151407 12645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "dffcbb35cc764339a85d8fe133c2142f"
format_stamp: "Formatted at 2026-08-07 19:44:57 on dist-test-slave-vq2q"
server_key: "1c8aaa6246a902139858732d543f44b1"
server_key_iv: "427480bb5d080b368637000b27096ba6"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:57.151707 12645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:57.179338 12645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:57.184567 12645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:57.184926 12645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:57.204412 12645 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:44429
I20260807 19:44:57.204435 12718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:44429 every 8 connection(s)
I20260807 19:44:57.206220 12645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:44:57.210072 12719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:44:57.213212 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 12645
I20260807 19:44:57.213447 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:44:57.214248 31552 external_mini_cluster.cc:1428] Setting key 36a080486c832839b27259077e156e9b
I20260807 19:44:57.224011 12719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f: Bootstrap starting.
I20260807 19:44:57.228617 12719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f: Neither blocks nor log segments found. Creating new log.
I20260807 19:44:57.230284 12719 log.cc:824] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:44:57.234695 12719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f: No bootstrap required, opened a new log
I20260807 19:44:57.244449 12719 raft_consensus.cc:348] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffcbb35cc764339a85d8fe133c2142f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44429 } }
I20260807 19:44:57.244989 12719 raft_consensus.cc:374] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:44:57.245150 12719 raft_consensus.cc:729] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dffcbb35cc764339a85d8fe133c2142f, State: Initialized, Role: FOLLOWER
I20260807 19:44:57.245889 12719 consensus_queue.cc:260] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffcbb35cc764339a85d8fe133c2142f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44429 } }
Aug 07 19:44:57 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:44:57.246122 12719 raft_consensus.cc:388] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:44:57.246249 12719 raft_consensus.cc:482] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:44:57.246440 12719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:44:57.249591 12719 raft_consensus.cc:504] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffcbb35cc764339a85d8fe133c2142f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44429 } }
I20260807 19:44:57.250249 12719 leader_election.cc:302] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dffcbb35cc764339a85d8fe133c2142f; no voters:
I20260807 19:44:57.250998 12719 leader_election.cc:288] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:44:57.251209 12724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:44:57.260032 12722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:57.216388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44206 (local address 127.30.208.62:44429)
0807 19:44:57.231220 (+ 14832us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:57.231238 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:57.231298 (+ 60us) server_negotiation.cc:407] Connection header received
0807 19:44:57.233964 (+ 2666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:57.233998 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:57.234515 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:57.235094 (+ 579us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:57.236759 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:57.238913 (+ 2154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:57.240927 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:57.242119 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:57.242314 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:57.247471 (+ 5157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:57.247484 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:57.247505 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:57.247563 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:57.251667 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:57.252840 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:57.252851 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:57.252864 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:57.253091 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:57.253653 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:57.253658 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:57.253662 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:57.254549 (+ 887us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:57.256074 (+ 1525us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:57.256419 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 19:44:57.256990 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14531,"thread_start_us":14792,"threads_started":1}
I20260807 19:44:57.262080 12724 raft_consensus.cc:686] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 1 LEADER]: Becoming Leader. State: Replica: dffcbb35cc764339a85d8fe133c2142f, State: Running, Role: LEADER
I20260807 19:44:57.262840 12724 consensus_queue.cc:237] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffcbb35cc764339a85d8fe133c2142f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44429 } }
I20260807 19:44:57.263252 12719 sys_catalog.cc:564] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:44:57.266430 12725 sys_catalog.cc:455] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dffcbb35cc764339a85d8fe133c2142f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffcbb35cc764339a85d8fe133c2142f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44429 } } }
I20260807 19:44:57.266974 12725 sys_catalog.cc:458] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:57.266408 12727 sys_catalog.cc:455] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dffcbb35cc764339a85d8fe133c2142f. Latest consensus state: current_term: 1 leader_uuid: "dffcbb35cc764339a85d8fe133c2142f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffcbb35cc764339a85d8fe133c2142f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44429 } } }
I20260807 19:44:57.273468 12727 sys_catalog.cc:458] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [sys.catalog]: This master's current role is: LEADER
I20260807 19:44:57.274034 12730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:44:57.280017 12730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:44:57.310480 12730 catalog_manager.cc:1347] Generated new cluster ID: abd05f8d5b7e49f1a8cace0b288aedf1
I20260807 19:44:57.310626 12730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:44:57.346688 12719 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 19:44:57.361013 12730 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:44:57.363049 12730 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:44:57.378939 12730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f: Generated new TSK 0
I20260807 19:44:57.380102 12730 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:44:58.141108 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.208.20:34907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44429
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:44:58 dist-test-slave-vq2q krb5kdc[12146](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131898, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 19:44:58.552033 12766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:58.552507 12766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:58.552700 12766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:58.564534 12766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:58.564682 12766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:58.564852 12766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:44:58.580700 12766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.208.20:34907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:44:58.582413 12766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:58.585291 12766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:58.602326 12777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:58.605909 12778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:58.611696 12766 server_base.cc:1034] running on GCE node
W20260807 19:44:58.612393 12780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:58.613338 12766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:58.614588 12766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:58.615850 12766 hybrid_clock.cc:648] HybridClock initialized: now 1786131898615765 us; error 80 us; skew 500 ppm
I20260807 19:44:58.616195 12766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:58 dist-test-slave-vq2q krb5kdc[12146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131898, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:58.619781 12766 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:44:58.621639 12766 webserver.cc:460] Webserver started at http://127.30.208.1:32899/ using document root <none> and password file <none>
I20260807 19:44:58.622663 12766 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:58.622876 12766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:58.623260 12766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:44:58 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131898, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:44:58.642866 12766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "9d83c79111ac42c98aed84711a5b3f8c"
format_stamp: "Formatted at 2026-08-07 19:44:58 on dist-test-slave-vq2q"
server_key: "9b410a8175c66660fe209f3cf5b590e7"
server_key_iv: "6a2e251ca5a20309e07be6b8945f9598"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:58.643996 12766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "9d83c79111ac42c98aed84711a5b3f8c"
format_stamp: "Formatted at 2026-08-07 19:44:58 on dist-test-slave-vq2q"
server_key: "9b410a8175c66660fe209f3cf5b590e7"
server_key_iv: "6a2e251ca5a20309e07be6b8945f9598"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:58.651335 12766 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260807 19:44:58.685093 12790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:58.687206 12766 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.003s sys 0.004s
I20260807 19:44:58.687425 12766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "9d83c79111ac42c98aed84711a5b3f8c"
format_stamp: "Formatted at 2026-08-07 19:44:58 on dist-test-slave-vq2q"
server_key: "9b410a8175c66660fe209f3cf5b590e7"
server_key_iv: "6a2e251ca5a20309e07be6b8945f9598"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:58.687721 12766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:58.708911 12766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:58.714296 12766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:58.714651 12766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:58.716261 12766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:58.724647 12766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:58.724751 12766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:58.724920 12766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:58.725071 12766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:58.794140 12766 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:44527
I20260807 19:44:58.794158 12915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:44527 every 8 connection(s)
I20260807 19:44:58.795810 12766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:44:58.796729 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 12766
I20260807 19:44:58.796936 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:44:58.797837 31552 external_mini_cluster.cc:1428] Setting key b16b20ab5fec4c4ad40ab516df9fbacd
Aug 07 19:44:58 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131898, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
Aug 07 19:44:58 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131898, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
I20260807 19:44:58.865614 12922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:58.829157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:44429 (local address 127.30.208.1:54185)
0807 19:44:58.837865 (+ 8708us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:58.837928 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:58.840510 (+ 2582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:58.844539 (+ 4029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:58.844554 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:58.845231 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:58.846076 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:58.846103 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.847691 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.847696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:58.849545 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:58.849574 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.851164 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.851171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:58.851475 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:58.852253 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:58.852281 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:58.855416 (+ 3135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:58.859999 (+ 4583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:58.860008 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:58.860026 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:58.860502 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:58.860864 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:58.860870 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:58.860873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:58.861112 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:58.861848 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:58.861856 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:58.862290 (+ 434us) client_negotiation.cc:769] Sending connection context
0807 19:44:58.862574 (+ 284us) client_negotiation.cc:240] Negotiation successful
0807 19:44:58.863118 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8308,"thread_start_us":8884,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
I20260807 19:44:58.866842 12920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:58.828533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:54185 (local address 127.30.208.62:44429)
0807 19:44:58.843776 (+ 15243us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:58.843782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:58.843804 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:44:58.843867 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:58.843882 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:58.843974 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:58.846268 (+ 2294us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:58.846439 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.847502 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.849773 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:58.850356 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:58.851921 (+ 1565us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:58.855728 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:58.855736 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:58.855741 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:58.855795 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:58.859818 (+ 4023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:58.860636 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:58.860639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:58.860642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:58.860754 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:58.861282 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:58.861287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:58.861290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:58.861640 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:58.866107 (+ 4467us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:58.866285 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 19:44:58.866507 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15102,"thread_start_us":15539,"threads_started":1}
I20260807 19:44:58.875820 12917 heartbeater.cc:347] Connected to a master server at 127.30.208.62:44429
I20260807 19:44:58.876199 12917 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:58.877231 12917 heartbeater.cc:511] Master 127.30.208.62:44429 requested a full tablet report, sending...
I20260807 19:44:58.879724 12681 ts_manager.cc:194] Registered new tserver with Master: 9d83c79111ac42c98aed84711a5b3f8c (127.30.208.1:44527)
I20260807 19:44:58.881376 12681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:54185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
W20260807 19:44:58.908620 31552 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.30.208.2: real 0.104s user 0.001s sys 0.022s
I20260807 19:44:58.909608 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.208.20:34907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44429
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:44:59 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131898, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
W20260807 19:44:59.385048 12924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:44:59.385545 12924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:44:59.385757 12924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:44:59.397461 12924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:44:59.397671 12924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:44:59.397917 12924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:44:59.413652 12924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.208.20:34907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:44:59.415491 12924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:44:59.417310 12924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:44:59.433625 12938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:59.433974 12941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:44:59.433589 12939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:44:59.435463 12924 server_base.cc:1034] running on GCE node
I20260807 19:44:59.436077 12924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:44:59.437057 12924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:44:59.438254 12924 hybrid_clock.cc:648] HybridClock initialized: now 1786131899438192 us; error 57 us; skew 500 ppm
I20260807 19:44:59.438566 12924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:44:59 dist-test-slave-vq2q krb5kdc[12146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131899, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:44:59.441641 12924 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:44:59.443207 12924 webserver.cc:460] Webserver started at http://127.30.208.2:38175/ using document root <none> and password file <none>
I20260807 19:44:59.443842 12924 fs_manager.cc:359] Metadata directory not provided
I20260807 19:44:59.443948 12924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:44:59.444231 12924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:44:59 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131899, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:44:59.461683 12924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "a699e133ce1748a29ef607a294290177"
format_stamp: "Formatted at 2026-08-07 19:44:59 on dist-test-slave-vq2q"
server_key: "4db9196ae9a5d85ba8675302ae051043"
server_key_iv: "b9a79466a504205a11b8687a7240662f"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:59.462553 12924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "a699e133ce1748a29ef607a294290177"
format_stamp: "Formatted at 2026-08-07 19:44:59 on dist-test-slave-vq2q"
server_key: "4db9196ae9a5d85ba8675302ae051043"
server_key_iv: "b9a79466a504205a11b8687a7240662f"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:59.468000 12924 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:44:59.484386 12948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:59.485918 12924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:44:59.486096 12924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "a699e133ce1748a29ef607a294290177"
format_stamp: "Formatted at 2026-08-07 19:44:59 on dist-test-slave-vq2q"
server_key: "4db9196ae9a5d85ba8675302ae051043"
server_key_iv: "b9a79466a504205a11b8687a7240662f"
server_key_version: "kuduclusterkey@0"
I20260807 19:44:59.486313 12924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:44:59.511888 12924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:44:59.516008 12924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:44:59.516331 12924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:44:59.517381 12924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:44:59.518844 12924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:44:59.518966 12924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:59.519116 12924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:44:59.519193 12924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:44:59.569644 12924 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:34267
I20260807 19:44:59.569660 13075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:34267 every 8 connection(s)
I20260807 19:44:59.571020 12924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:44:59.577464 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 12924
I20260807 19:44:59.577739 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:44:59.578372 31552 external_mini_cluster.cc:1428] Setting key 67933340c38ff271824d7928842f3a69
Aug 07 19:44:59 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131899, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:44:59.605971 13081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:59.574655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:44429 (local address 127.30.208.2:51463)
0807 19:44:59.582265 (+ 7610us) negotiation.cc:107] Waiting for socket to connect
0807 19:44:59.582317 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:44:59.585046 (+ 2729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:44:59.586186 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:44:59.586209 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:44:59.586814 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:44:59.587655 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:59.587695 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.589278 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.589283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:59.591302 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:44:59.591318 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.592570 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.592578 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:44:59.592890 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:59.593681 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:44:59.593760 (+ 79us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:44:59.596509 (+ 2749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:44:59.601129 (+ 4620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:59.601137 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:59.601154 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:59.601776 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:59.602311 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:44:59.602316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:44:59.602319 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:44:59.602579 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:44:59.603292 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:44:59.603300 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:44:59.603709 (+ 409us) client_negotiation.cc:769] Sending connection context
0807 19:44:59.604008 (+ 299us) client_negotiation.cc:240] Negotiation successful
0807 19:44:59.604518 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7294,"thread_start_us":7085,"threads_started":1}
I20260807 19:44:59.607029 13082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:44:59.575328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:51463 (local address 127.30.208.62:44429)
0807 19:44:59.581775 (+ 6447us) server_negotiation.cc:206] Beginning negotiation
0807 19:44:59.581782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:44:59.582844 (+ 1062us) server_negotiation.cc:407] Connection header received
0807 19:44:59.585406 (+ 2562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:44:59.585412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:44:59.585497 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:44:59.585578 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:44:59.587906 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.589041 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.591471 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:44:59.592124 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:44:59.592225 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:44:59.596759 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:44:59.596766 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:44:59.596771 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:44:59.596812 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:44:59.600965 (+ 4153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:59.602029 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:59.602035 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:59.602038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:59.602186 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:44:59.602710 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:44:59.602715 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:44:59.602719 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:44:59.603083 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:44:59.603222 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 19:44:59.606446 (+ 3224us) server_negotiation.cc:299] Negotiation successful
0807 19:44:59.606674 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6279,"thread_start_us":6139,"threads_started":1}
I20260807 19:44:59.609069 13078 heartbeater.cc:347] Connected to a master server at 127.30.208.62:44429
I20260807 19:44:59.609404 13078 heartbeater.cc:464] Registering TS with master...
I20260807 19:44:59.610211 13078 heartbeater.cc:511] Master 127.30.208.62:44429 requested a full tablet report, sending...
I20260807 19:44:59.611835 12681 ts_manager.cc:194] Registered new tserver with Master: a699e133ce1748a29ef607a294290177 (127.30.208.2:34267)
I20260807 19:44:59.612900 12681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:51463
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:44:59.658386 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.208.20:34907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44429
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:44:59.884999 12917 heartbeater.cc:503] Master 127.30.208.62:44429 was elected leader, sending a full tablet report...
W20260807 19:45:00.033388 13085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:00.033682 13085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:00.033790 13085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:00.041077 13085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:00.041220 13085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:00.041363 13085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:45:00.051359 13085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.208.20:34907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:00.052577 13085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:00.053853 13085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:00.066428 13090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:00.066509 13091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:00.066902 13094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:00.067292 13085 server_base.cc:1034] running on GCE node
I20260807 19:45:00.067837 13085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:00.068722 13085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:00.069921 13085 hybrid_clock.cc:648] HybridClock initialized: now 1786131900069870 us; error 43 us; skew 500 ppm
I20260807 19:45:00.070214 13085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:00 dist-test-slave-vq2q krb5kdc[12146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:00.073254 13085 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:45:00.074616 13085 webserver.cc:460] Webserver started at http://127.30.208.3:43685/ using document root <none> and password file <none>
I20260807 19:45:00.075264 13085 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:00.075382 13085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:00.075678 13085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 19:45:00 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:45:00.091848 13085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e"
format_stamp: "Formatted at 2026-08-07 19:45:00 on dist-test-slave-vq2q"
server_key: "acde79c694e9f751aaf1034a2b12b861"
server_key_iv: "1e7b6c72b7c69098bf8cce55a5327816"
server_key_version: "kuduclusterkey@0"
I20260807 19:45:00.092705 13085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e"
format_stamp: "Formatted at 2026-08-07 19:45:00 on dist-test-slave-vq2q"
server_key: "acde79c694e9f751aaf1034a2b12b861"
server_key_iv: "1e7b6c72b7c69098bf8cce55a5327816"
server_key_version: "kuduclusterkey@0"
I20260807 19:45:00.098284 13085 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:45:00.113224 13101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:00.114711 13085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:45:00.114882 13085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e"
format_stamp: "Formatted at 2026-08-07 19:45:00 on dist-test-slave-vq2q"
server_key: "acde79c694e9f751aaf1034a2b12b861"
server_key_iv: "1e7b6c72b7c69098bf8cce55a5327816"
server_key_version: "kuduclusterkey@0"
I20260807 19:45:00.115096 13085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:00.137228 13085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:00.141255 13085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:00.141553 13085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:00.142529 13085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:00.143955 13085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:00.144037 13085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:00.144155 13085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:00.144207 13085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:00.174723 13085 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:46143
I20260807 19:45:00.174789 13229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:46143 every 8 connection(s)
I20260807 19:45:00.175984 13085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:45:00.178314 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 13085
I20260807 19:45:00.178521 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:45:00.179131 31552 external_mini_cluster.cc:1428] Setting key 86f453ecbec3dd7b80db29600138924b
Aug 07 19:45:00 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:00.196727 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:00.178625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:36623 (local address 127.30.208.62:44429)
0807 19:45:00.179156 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:00.179161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:00.180335 (+ 1174us) server_negotiation.cc:407] Connection header received
0807 19:45:00.182306 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:00.182310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:00.182386 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:00.182497 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:00.183862 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.184571 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.186215 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.186709 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.186833 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:00.190530 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:00.190535 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:00.190539 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:00.190575 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:00.193775 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:00.194563 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:00.194567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:00.194569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:00.194776 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:00.195193 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:00.195197 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:00.195199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:00.195469 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:00.195579 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:00.196240 (+ 661us) server_negotiation.cc:299] Negotiation successful
0807 19:45:00.196420 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":255,"threads_started":1}
I20260807 19:45:00.197737 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:00.179296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:44429 (local address 127.30.208.3:36623)
0807 19:45:00.180174 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:00.180224 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:00.182043 (+ 1819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:00.182674 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:00.182681 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:00.183135 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:00.183704 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:00.183722 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.184763 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.184767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:00.186081 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:00.186100 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.186976 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.186983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:00.187192 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:00.187819 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:00.187848 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:00.190301 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:00.193925 (+ 3624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:00.193933 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:00.193950 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:00.194442 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:00.194885 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:00.194888 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:00.194890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:00.195084 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:00.195559 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:00.195565 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:00.195852 (+ 287us) client_negotiation.cc:769] Sending connection context
0807 19:45:00.196109 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 19:45:00.196475 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":389,"threads_started":1}
I20260807 19:45:00.199115 13231 heartbeater.cc:347] Connected to a master server at 127.30.208.62:44429
I20260807 19:45:00.199386 13231 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:00.200071 13231 heartbeater.cc:511] Master 127.30.208.62:44429 requested a full tablet report, sending...
I20260807 19:45:00.201551 12681 ts_manager.cc:194] Registered new tserver with Master: ad242fb73aec4cc49fcdc3f7a5483e5e (127.30.208.3:46143)
I20260807 19:45:00.202566 12681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:36623
I20260807 19:45:00.206861 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:45:00.208877 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 12766
Aug 07 19:45:00 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:45:00.229897 13237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:00.209822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57724 (local address 127.30.208.1:44527)
0807 19:45:00.213141 (+ 3319us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:00.213147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:00.213172 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:45:00.213261 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:00.213266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:00.213472 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:00.213599 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:00.215436 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.216525 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.218313 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.219002 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.219097 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:00.223997 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:00.224011 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:00.224026 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:00.224064 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:00.227262 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:00.227920 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:00.227927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:00.227929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:00.228082 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:00.228417 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:00.228420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:00.228422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:00.228738 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:00.229014 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:00.229275 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 19:45:00.229489 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3185,"thread_start_us":3048,"threads_started":1}
I20260807 19:45:00.231297 12911 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:00.315047 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 12766
I20260807 19:45:00.331347 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 12924
Aug 07 19:45:00 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:45:00.347136 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:00.332243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54230 (local address 127.30.208.2:34267)
0807 19:45:00.332738 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:00.332745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:00.332768 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:45:00.332828 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:00.332834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:00.333053 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:00.333219 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:00.334640 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.335683 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.337150 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.337772 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.337897 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:00.342088 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:00.342099 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:00.342112 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:00.342144 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:00.344739 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:00.345424 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:00.345431 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:00.345434 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:00.345559 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:00.345850 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:00.345852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:00.345854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:00.346199 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:00.346431 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:00.346640 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 19:45:00.346836 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":250,"threads_started":1}
I20260807 19:45:00.348351 13070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:00.430126 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 12924
I20260807 19:45:00.445890 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 13085
Aug 07 19:45:00 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:45:00.463136 13241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:00.446854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49916 (local address 127.30.208.3:46143)
0807 19:45:00.447473 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:00.447480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:00.447509 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:45:00.447567 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:00.447574 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:00.447783 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:00.447919 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:00.449535 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.450614 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.452110 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:00.452690 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:00.452802 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:00.457360 (+ 4558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:00.457371 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:00.457387 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:00.457422 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:00.460335 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:00.461012 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:00.461019 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:00.461022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:00.461158 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:00.461557 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:00.461561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:00.461563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:00.461919 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:00.462229 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:00.462531 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 19:45:00.462789 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":307,"threads_started":1}
I20260807 19:45:00.464581 13225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:00.546113 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 13085
I20260807 19:45:00.562531 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 12645
I20260807 19:45:00.563788 12713 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:00.669348 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 12645
I20260807 19:45:00.695327 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:44429
--webserver_interface=127.30.208.62
--webserver_port=43669
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:44429
--ranger_config_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.208.20:34907/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.208.20:34907/kms with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:01.022444 13244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:01.022740 13244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:01.022845 13244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:01.030121 13244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:45:01.030227 13244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:01.030277 13244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:01.030331 13244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:45:01.030378 13244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:45:01.040572 13244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.208.20:34907/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:44429
--ranger_config_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:44429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=43669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:01.041877 13244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:01.043164 13244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:01.054968 13249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:01.055012 13250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:01.054945 13252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:01.055790 13244 server_base.cc:1034] running on GCE node
I20260807 19:45:01.056576 13244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:01.058059 13244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:01.059275 13244 hybrid_clock.cc:648] HybridClock initialized: now 1786131901059227 us; error 41 us; skew 500 ppm
I20260807 19:45:01.059612 13244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:01 dist-test-slave-vq2q krb5kdc[12146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:01.062587 13244 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:45:01.063974 13244 webserver.cc:460] Webserver started at http://127.30.208.62:43669/ using document root <none> and password file <none>
I20260807 19:45:01.064571 13244 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:01.064769 13244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:45:01 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:45:01.146888 13244 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:45:01.150048 13262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:01.151482 13244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:45:01.151652 13244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "dffcbb35cc764339a85d8fe133c2142f"
format_stamp: "Formatted at 2026-08-07 19:44:57 on dist-test-slave-vq2q"
server_key: "1c8aaa6246a902139858732d543f44b1"
server_key_iv: "427480bb5d080b368637000b27096ba6"
server_key_version: "kuduclusterkey@0"
I20260807 19:45:01.152624 13244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:01.170083 13244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:01.173669 13244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:01.173946 13244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:01.192077 13244 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:44429
I20260807 19:45:01.192101 13319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:44429 every 8 connection(s)
I20260807 19:45:01.193650 13244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:45:01.200465 13320 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:45:01.202353 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 13244
I20260807 19:45:01.203398 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.208.20:34907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:44527
--local_ip_for_outbound_sockets=127.30.208.1
--tserver_master_addrs=127.30.208.62:44429
--webserver_port=32899
--webserver_interface=127.30.208.1
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.208.20:34907/kms with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:45:01.204550 13320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f: Bootstrap starting.
I20260807 19:45:01.227766 13320 log.cc:824] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:01.236948 13320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:45:01.237528 13320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f: Bootstrap complete.
I20260807 19:45:01.247170 13320 raft_consensus.cc:348] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffcbb35cc764339a85d8fe133c2142f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44429 } }
I20260807 19:45:01.248067 13320 raft_consensus.cc:729] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dffcbb35cc764339a85d8fe133c2142f, State: Initialized, Role: FOLLOWER
I20260807 19:45:01.248800 13320 consensus_queue.cc:260] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffcbb35cc764339a85d8fe133c2142f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44429 } }
I20260807 19:45:01.249114 13320 raft_consensus.cc:388] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:45:01.249259 13320 raft_consensus.cc:482] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:45:01.249434 13320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:45:01.253417 13320 raft_consensus.cc:504] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffcbb35cc764339a85d8fe133c2142f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44429 } }
I20260807 19:45:01.253939 13320 leader_election.cc:302] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dffcbb35cc764339a85d8fe133c2142f; no voters:
I20260807 19:45:01.254513 13320 leader_election.cc:288] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:45:01.254714 13324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:45:01.255853 13324 raft_consensus.cc:686] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [term 2 LEADER]: Becoming Leader. State: Replica: dffcbb35cc764339a85d8fe133c2142f, State: Running, Role: LEADER
I20260807 19:45:01.256578 13324 consensus_queue.cc:237] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffcbb35cc764339a85d8fe133c2142f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44429 } }
I20260807 19:45:01.256717 13320 sys_catalog.cc:564] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:45:01.259348 13326 sys_catalog.cc:455] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dffcbb35cc764339a85d8fe133c2142f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffcbb35cc764339a85d8fe133c2142f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44429 } } }
I20260807 19:45:01.259768 13326 sys_catalog.cc:458] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:01.264535 13327 sys_catalog.cc:455] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dffcbb35cc764339a85d8fe133c2142f. Latest consensus state: current_term: 2 leader_uuid: "dffcbb35cc764339a85d8fe133c2142f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dffcbb35cc764339a85d8fe133c2142f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44429 } } }
I20260807 19:45:01.264865 13327 sys_catalog.cc:458] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:01.268358 13330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:45:01.275977 13330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:45:01.277657 13330 catalog_manager.cc:1259] Loaded cluster ID: abd05f8d5b7e49f1a8cace0b288aedf1
I20260807 19:45:01.277855 13330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:45:01.282366 13330 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:45:01.284089 13330 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f: Loaded TSK: 0
I20260807 19:45:01.286036 13330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:45:01.289113 13320 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260807 19:45:01.684757 13322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:01.685463 13322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:01.685698 13322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:01.697275 13322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:01.697468 13322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:01.697654 13322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:45:01.713248 13322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.208.20:34907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:44527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=32899
--tserver_master_addrs=127.30.208.62:44429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:01.715265 13322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:01.717111 13322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:01.751289 13363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:01.757619 13366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:01.758101 13322 server_base.cc:1034] running on GCE node
W20260807 19:45:01.758771 13362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:01.759694 13322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:01.766759 13322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:01.768142 13322 hybrid_clock.cc:648] HybridClock initialized: now 1786131901768076 us; error 64 us; skew 500 ppm
I20260807 19:45:01.768689 13322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:01 dist-test-slave-vq2q krb5kdc[12146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:01.773113 13322 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:45:01.783125 13322 webserver.cc:460] Webserver started at http://127.30.208.1:32899/ using document root <none> and password file <none>
I20260807 19:45:01.784158 13322 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:01.784394 13322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:45:01 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:45:01.825551 13322 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:45:01.836536 13375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:01.838248 13322 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.002s sys 0.006s
I20260807 19:45:01.838439 13322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "9d83c79111ac42c98aed84711a5b3f8c"
format_stamp: "Formatted at 2026-08-07 19:44:58 on dist-test-slave-vq2q"
server_key: "9b410a8175c66660fe209f3cf5b590e7"
server_key_iv: "6a2e251ca5a20309e07be6b8945f9598"
server_key_version: "kuduclusterkey@0"
I20260807 19:45:01.839461 13322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:01.866132 13322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:01.870280 13322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:01.870558 13322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:01.871578 13322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:01.872992 13322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:01.873088 13322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:01.873226 13322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:01.873306 13322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:01.935144 13322 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:44527
I20260807 19:45:01.935187 13502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:44527 every 8 connection(s)
I20260807 19:45:01.936769 13322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:45:01.938220 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 13322
I20260807 19:45:01.939034 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.208.20:34907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:34267
--local_ip_for_outbound_sockets=127.30.208.2
--tserver_master_addrs=127.30.208.62:44429
--webserver_port=38175
--webserver_interface=127.30.208.2
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.208.20:34907/kms with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:45:01 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:01.981657 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:01.947881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:44429 (local address 127.30.208.1:35531)
0807 19:45:01.956012 (+ 8131us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:01.956064 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:01.958553 (+ 2489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:01.961003 (+ 2450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:01.961014 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:01.961657 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:01.962408 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:01.962432 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.964666 (+ 2234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.964670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:01.966492 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:01.966508 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.968178 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.968184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:01.968440 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:01.969257 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:01.969289 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:01.972100 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:01.976532 (+ 4432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:01.976541 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:01.976558 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:01.977149 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:01.977723 (+ 574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:01.977728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:01.977732 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:01.977972 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:01.979027 (+ 1055us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:01.979035 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:01.979434 (+ 399us) client_negotiation.cc:769] Sending connection context
0807 19:45:01.979720 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 19:45:01.980293 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7848,"thread_start_us":7617,"threads_started":1}
I20260807 19:45:01.983837 13506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:01.948950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:35531 (local address 127.30.208.62:44429)
0807 19:45:01.953771 (+ 4821us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:01.953794 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:01.956414 (+ 2620us) server_negotiation.cc:407] Connection header received
0807 19:45:01.959288 (+ 2874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:01.959310 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:01.959786 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:01.960142 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:01.962665 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.964450 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.966648 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:01.967613 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:01.967764 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:01.972452 (+ 4688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:01.972464 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:01.972486 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:01.972541 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:01.976317 (+ 3776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:01.977359 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:01.977369 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:01.977388 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:01.977583 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:01.978099 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:01.978104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:01.978108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:01.978763 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:01.978956 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:01.982286 (+ 3330us) server_negotiation.cc:299] Negotiation successful
0807 19:45:01.982704 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4549,"thread_start_us":4353,"threads_started":1}
I20260807 19:45:01.988091 13503 heartbeater.cc:347] Connected to a master server at 127.30.208.62:44429
I20260807 19:45:01.988408 13503 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:01.989207 13503 heartbeater.cc:511] Master 127.30.208.62:44429 requested a full tablet report, sending...
W20260807 19:45:01.992395 13503 heartbeater.cc:644] Failed to heartbeat to 127.30.208.62:44429 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 19:45:01.992612 13503 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:01.992874 13503 heartbeater.cc:511] Master 127.30.208.62:44429 requested a full tablet report, sending...
I20260807 19:45:01.994182 13503 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:01.994522 13503 heartbeater.cc:511] Master 127.30.208.62:44429 requested a full tablet report, sending...
W20260807 19:45:01.995594 13503 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:45:02.351449 13507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:02.351742 13507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:02.351913 13507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:02.359553 13507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:02.359675 13507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:02.359849 13507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:45:02.370024 13507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.208.20:34907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:34267
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=38175
--tserver_master_addrs=127.30.208.62:44429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:02.371559 13507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:02.373370 13507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:02.411026 13521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:02.411711 13522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:02.433323 13507 server_base.cc:1034] running on GCE node
W20260807 19:45:02.434687 13526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:02.435700 13507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:02.443358 13507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:02.444617 13507 hybrid_clock.cc:648] HybridClock initialized: now 1786131902444554 us; error 57 us; skew 500 ppm
I20260807 19:45:02.444983 13507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:02 dist-test-slave-vq2q krb5kdc[12146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131902, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:02.454849 13507 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:45:02.457093 13507 webserver.cc:460] Webserver started at http://127.30.208.2:38175/ using document root <none> and password file <none>
I20260807 19:45:02.457983 13507 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:02.458122 13507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:45:02 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131902, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:45:02.494853 13507 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.007s sys 0.003s
I20260807 19:45:02.498041 13534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:02.499521 13507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:45:02.499680 13507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "a699e133ce1748a29ef607a294290177"
format_stamp: "Formatted at 2026-08-07 19:44:59 on dist-test-slave-vq2q"
server_key: "4db9196ae9a5d85ba8675302ae051043"
server_key_iv: "b9a79466a504205a11b8687a7240662f"
server_key_version: "kuduclusterkey@0"
I20260807 19:45:02.500669 13507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 07 19:45:02 dist-test-slave-vq2q krb5kdc[12146](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786131902, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:02.531471 13507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:02.536648 13507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:02.537024 13507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:02.539960 13507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:02.544412 13507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:02.544518 13507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:02.544634 13507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:02.544770 13507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 19:45:02.590132 13507 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:34267
I20260807 19:45:02.590315 13663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:34267 every 8 connection(s)
I20260807 19:45:02.591677 13507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:45:02.600658 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 13507
I20260807 19:45:02.601488 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.208.20:34907/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:46143
--local_ip_for_outbound_sockets=127.30.208.3
--tserver_master_addrs=127.30.208.62:44429
--webserver_port=43685
--webserver_interface=127.30.208.3
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.208.20:34907/kms with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:45:02 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131902, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:02.632040 13667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:02.595314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:44429 (local address 127.30.208.2:45929)
0807 19:45:02.598195 (+ 2881us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:02.598243 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:02.600739 (+ 2496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:02.611595 (+ 10856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:02.611610 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:02.612348 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:02.613308 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:02.613340 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.614929 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.614935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:02.617027 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:02.617045 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.618259 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.618268 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:02.618573 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:02.619496 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:02.619538 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:02.622637 (+ 3099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:02.626908 (+ 4271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:02.626918 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:02.626939 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:02.627599 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:02.628099 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:02.628104 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:02.628107 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:02.628416 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:02.629116 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:02.629124 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:02.629629 (+ 505us) client_negotiation.cc:769] Sending connection context
0807 19:45:02.629954 (+ 325us) client_negotiation.cc:240] Negotiation successful
0807 19:45:02.630552 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2586,"thread_start_us":2371,"threads_started":1}
I20260807 19:45:02.634003 13666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:02.594594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:45929 (local address 127.30.208.62:44429)
0807 19:45:02.610865 (+ 16271us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:02.610872 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:02.610901 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:45:02.610977 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:02.610982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:02.611068 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:02.611212 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:02.613597 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.614707 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.617287 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:02.617978 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:02.619156 (+ 1178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:02.622942 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:02.622950 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:02.622954 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:02.622996 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:02.626719 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:02.627790 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:02.627797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:02.627801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:02.627965 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:02.628574 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:02.628578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:02.628581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:02.628952 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:02.633233 (+ 4281us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:02.633385 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 19:45:02.633645 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16126,"thread_start_us":15985,"threads_started":1}
I20260807 19:45:02.636075 13664 heartbeater.cc:347] Connected to a master server at 127.30.208.62:44429
I20260807 19:45:02.636439 13664 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:02.637372 13664 heartbeater.cc:511] Master 127.30.208.62:44429 requested a full tablet report, sending...
I20260807 19:45:02.639683 13280 ts_manager.cc:194] Registered new tserver with Master: a699e133ce1748a29ef607a294290177 (127.30.208.2:34267)
I20260807 19:45:02.641848 13280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:45929
Aug 07 19:45:02 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131902, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:45:02.996304 13503 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:02.997160 13503 heartbeater.cc:511] Master 127.30.208.62:44429 requested a full tablet report, sending...
I20260807 19:45:02.998845 13280 ts_manager.cc:194] Registered new tserver with Master: 9d83c79111ac42c98aed84711a5b3f8c (127.30.208.1:44527)
I20260807 19:45:03.000031 13280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:35531
W20260807 19:45:03.054618 13668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:03.055401 13668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:03.055595 13668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:03.068076 13668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:03.068269 13668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:03.068522 13668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:45:03.084826 13668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.208.20:34907/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:46143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=43685
--tserver_master_addrs=127.30.208.62:44429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:03.086747 13668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:03.088595 13668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:03.103703 13675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:03.103801 13678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:03.104344 13674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:03.107307 13668 server_base.cc:1034] running on GCE node
I20260807 19:45:03.108103 13668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:03.109570 13668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:03.110831 13668 hybrid_clock.cc:648] HybridClock initialized: now 1786131903110774 us; error 35 us; skew 500 ppm
I20260807 19:45:03.111207 13668 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:03 dist-test-slave-vq2q krb5kdc[12146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:03.115442 13668 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:45:03.117502 13668 webserver.cc:460] Webserver started at http://127.30.208.3:43685/ using document root <none> and password file <none>
I20260807 19:45:03.118441 13668 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:03.118606 13668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 19:45:03 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:45:03.147161 13668 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.007s sys 0.000s
I20260807 19:45:03.150861 13687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:03.152626 13668 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:45:03.152863 13668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e"
format_stamp: "Formatted at 2026-08-07 19:45:00 on dist-test-slave-vq2q"
server_key: "acde79c694e9f751aaf1034a2b12b861"
server_key_iv: "1e7b6c72b7c69098bf8cce55a5327816"
server_key_version: "kuduclusterkey@0"
I20260807 19:45:03.154407 13668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:03.189785 13668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:03.194890 13668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:03.195175 13668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:03.202632 13668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:03.204571 13668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:03.204752 13668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:03.204887 13668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:03.204967 13668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 19:45:03 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131902, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for HTTP/127.30.208.20@KRBTEST.COM
I20260807 19:45:03.295663 13668 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:46143
I20260807 19:45:03.295744 13811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:46143 every 8 connection(s)
I20260807 19:45:03.298760 13668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:45:03.304798 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 13668
Aug 07 19:45:03 dist-test-slave-vq2q krb5kdc[12146](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:03.354719 13814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.304112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:44429 (local address 127.30.208.3:42267)
0807 19:45:03.304917 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:03.304971 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:03.307282 (+ 2311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:03.332038 (+ 24756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:03.332055 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:03.332851 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:03.333890 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.333919 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.335768 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.335774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.338010 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.338031 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.339357 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.339365 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.339684 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.340542 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:03.340576 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:03.343659 (+ 3083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:03.349451 (+ 5792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:03.349462 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:03.349483 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:03.350178 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:03.350631 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:03.350637 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:03.350640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:03.350941 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:03.351610 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:03.351619 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:03.352100 (+ 481us) client_negotiation.cc:769] Sending connection context
0807 19:45:03.352418 (+ 318us) client_negotiation.cc:240] Negotiation successful
0807 19:45:03.353029 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":4510,"threads_started":1}
I20260807 19:45:03.356866 13817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.305611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:42267 (local address 127.30.208.62:44429)
0807 19:45:03.330562 (+ 24951us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:03.330570 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:03.330603 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:45:03.330700 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:03.330706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:03.330807 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:03.334131 (+ 3324us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:03.334322 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.335557 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.338256 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.338963 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.340164 (+ 1201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.344668 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:03.344674 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:03.344678 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:03.344713 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:03.349278 (+ 4565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:03.350348 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:03.350354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:03.350358 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:03.350506 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:03.351096 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:03.351100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:03.351103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:03.351458 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:03.355974 (+ 4516us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:03.356143 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 19:45:03.356460 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24766,"thread_start_us":33404,"threads_started":1}
I20260807 19:45:03.358786 13812 heartbeater.cc:347] Connected to a master server at 127.30.208.62:44429
I20260807 19:45:03.359184 13812 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:03.360357 13812 heartbeater.cc:511] Master 127.30.208.62:44429 requested a full tablet report, sending...
I20260807 19:45:03.362211 13280 ts_manager.cc:194] Registered new tserver with Master: ad242fb73aec4cc49fcdc3f7a5483e5e (127.30.208.3:46143)
I20260807 19:45:03.363417 13280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:42267
I20260807 19:45:03.367991 13815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.306061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40866 (local address 127.30.208.62:44429)
0807 19:45:03.331496 (+ 25435us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:03.331502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:03.344348 (+ 12846us) server_negotiation.cc:407] Connection header received
0807 19:45:03.344535 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:03.344541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:03.344629 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:03.344727 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:03.346950 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.348453 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.351169 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.351818 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.351928 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.355764 (+ 3836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:03.355771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:03.355776 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:03.355816 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:03.359496 (+ 3680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:03.360246 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:03.360252 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:03.360255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:03.360428 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:03.364892 (+ 4464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:03.364962 (+ 70us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:03.364967 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:03.365374 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:03.365679 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:03.367339 (+ 1660us) server_negotiation.cc:299] Negotiation successful
0807 19:45:03.367601 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25307,"thread_start_us":24957,"threads_started":1}
I20260807 19:45:03.371141 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:45:03.392957 13815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.374883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40882 (local address 127.30.208.62:44429)
0807 19:45:03.375068 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:03.375074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:03.375622 (+ 548us) server_negotiation.cc:407] Connection header received
0807 19:45:03.375814 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:03.375820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:03.375921 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:03.376024 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:03.378175 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.379305 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.381405 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.382050 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.382149 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.385735 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:03.385742 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:03.385747 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:03.385794 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:03.389431 (+ 3637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:03.390373 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:03.390380 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:03.390384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:03.390564 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:03.391085 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:03.391091 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:03.391095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:03.391477 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:03.391725 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:03.392353 (+ 628us) server_negotiation.cc:299] Negotiation successful
0807 19:45:03.392623 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:45:03.398825 13280 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40882:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:45:03.410276 13280 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:45:03.466353 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.443774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:34267 (local address 127.0.0.1:54242)
0807 19:45:03.455653 (+ 11879us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:03.455699 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:03.456065 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:03.457274 (+ 1209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:03.457283 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:03.457334 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:03.457785 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.457799 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.460021 (+ 2222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.460027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.462923 (+ 2896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.462942 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.465239 (+ 2297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.465247 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.465478 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.465539 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 19:45:03.465687 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 19:45:03.465913 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11617,"thread_start_us":11465,"threads_started":1}
I20260807 19:45:03.467365 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.444342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54242 (local address 127.30.208.2:34267)
0807 19:45:03.454811 (+ 10469us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:03.454822 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:03.455889 (+ 1067us) server_negotiation.cc:407] Connection header received
0807 19:45:03.456366 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:03.456374 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:03.456627 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:03.456856 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:03.458029 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.459806 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.463141 (+ 3335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.464674 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.464806 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.465039 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:03.466803 (+ 1764us) server_negotiation.cc:299] Negotiation successful
0807 19:45:03.467018 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10244,"thread_start_us":10501,"threads_started":1}
I20260807 19:45:03.481817 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.456479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:46143 (local address 127.0.0.1:49922)
0807 19:45:03.468044 (+ 11565us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:03.468065 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:03.468188 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:03.474533 (+ 6345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:03.474539 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:03.474574 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:03.474794 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.474806 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.477163 (+ 2357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.477169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.479027 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.479044 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.480999 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.481006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.481221 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.481263 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 19:45:03.481331 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 19:45:03.481494 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11426,"thread_start_us":20894,"threads_started":1}
I20260807 19:45:03.483253 13831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.456604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49922 (local address 127.30.208.3:46143)
0807 19:45:03.473785 (+ 17181us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:03.473796 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:03.473836 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:45:03.473981 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:03.473990 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:03.474286 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:03.474990 (+ 704us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:03.475285 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.476991 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.479247 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.480785 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.482310 (+ 1525us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.482587 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:03.482664 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 19:45:03.482848 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16970,"thread_start_us":16779,"threads_started":1}
I20260807 19:45:03.491263 13741 tablet_service.cc:1467] Processing CreateTablet for tablet c161aea837f04675976fb35b73610534 (DEFAULT_TABLE table=test-table [id=378fdbac2f6b428aa0310e973a3feefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:03.493011 13741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c161aea837f04675976fb35b73610534. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:03.502255 13590 tablet_service.cc:1467] Processing CreateTablet for tablet c161aea837f04675976fb35b73610534 (DEFAULT_TABLE table=test-table [id=378fdbac2f6b428aa0310e973a3feefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:03.503865 13590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c161aea837f04675976fb35b73610534. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:03.514452 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.484808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44527 (local address 127.0.0.1:57730)
0807 19:45:03.485159 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:03.485201 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:03.485312 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:03.506354 (+ 21042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:03.506360 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:03.506396 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:03.506600 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.506616 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.508804 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.508811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.511001 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.511021 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.513574 (+ 2553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.513581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.513805 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.513849 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 19:45:03.513932 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 19:45:03.514099 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241}
I20260807 19:45:03.516155 13832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.478465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57730 (local address 127.30.208.1:44527)
0807 19:45:03.505682 (+ 27217us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:03.505691 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:03.505722 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:45:03.505827 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:03.505834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:03.506094 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:03.506800 (+ 706us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:03.507064 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.508592 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.511256 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.513358 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.514828 (+ 1470us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.515236 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:03.515350 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 19:45:03.515626 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26751,"thread_start_us":26853,"threads_started":1}
I20260807 19:45:03.520455 13427 tablet_service.cc:1467] Processing CreateTablet for tablet c161aea837f04675976fb35b73610534 (DEFAULT_TABLE table=test-table [id=378fdbac2f6b428aa0310e973a3feefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:03.521162 13836 tablet_bootstrap.cc:492] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e: Bootstrap starting.
I20260807 19:45:03.522192 13427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c161aea837f04675976fb35b73610534. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:03.523092 13840 tablet_bootstrap.cc:492] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177: Bootstrap starting.
I20260807 19:45:03.525034 13836 tablet_bootstrap.cc:654] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:03.526361 13836 log.cc:824] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:03.527251 13840 tablet_bootstrap.cc:654] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:03.528589 13840 log.cc:824] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:03.530299 13836 tablet_bootstrap.cc:492] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e: No bootstrap required, opened a new log
I20260807 19:45:03.530601 13836 ts_tablet_manager.cc:1397] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 19:45:03.536474 13840 tablet_bootstrap.cc:492] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177: No bootstrap required, opened a new log
I20260807 19:45:03.536783 13840 ts_tablet_manager.cc:1397] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.006s
I20260807 19:45:03.539583 13836 raft_consensus.cc:348] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a699e133ce1748a29ef607a294290177" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34267 } } peers { permanent_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 46143 } } peers { permanent_uuid: "9d83c79111ac42c98aed84711a5b3f8c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44527 } }
I20260807 19:45:03.539959 13836 raft_consensus.cc:374] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:03.540055 13836 raft_consensus.cc:729] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad242fb73aec4cc49fcdc3f7a5483e5e, State: Initialized, Role: FOLLOWER
I20260807 19:45:03.540652 13836 consensus_queue.cc:260] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a699e133ce1748a29ef607a294290177" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34267 } } peers { permanent_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 46143 } } peers { permanent_uuid: "9d83c79111ac42c98aed84711a5b3f8c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44527 } }
I20260807 19:45:03.541787 13812 heartbeater.cc:503] Master 127.30.208.62:44429 was elected leader, sending a full tablet report...
I20260807 19:45:03.542833 13847 tablet_bootstrap.cc:492] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c: Bootstrap starting.
I20260807 19:45:03.543987 13836 ts_tablet_manager.cc:1428] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260807 19:45:03.546804 13847 tablet_bootstrap.cc:654] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:03.546949 13840 raft_consensus.cc:348] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a699e133ce1748a29ef607a294290177" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34267 } } peers { permanent_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 46143 } } peers { permanent_uuid: "9d83c79111ac42c98aed84711a5b3f8c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44527 } }
I20260807 19:45:03.547386 13840 raft_consensus.cc:374] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:03.547490 13840 raft_consensus.cc:729] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a699e133ce1748a29ef607a294290177, State: Initialized, Role: FOLLOWER
I20260807 19:45:03.548125 13840 consensus_queue.cc:260] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a699e133ce1748a29ef607a294290177" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34267 } } peers { permanent_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 46143 } } peers { permanent_uuid: "9d83c79111ac42c98aed84711a5b3f8c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44527 } }
I20260807 19:45:03.548486 13847 log.cc:824] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:03.549887 13664 heartbeater.cc:503] Master 127.30.208.62:44429 was elected leader, sending a full tablet report...
I20260807 19:45:03.550441 13840 ts_tablet_manager.cc:1428] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177: Time spent starting tablet: real 0.014s user 0.008s sys 0.007s
I20260807 19:45:03.551810 13847 tablet_bootstrap.cc:492] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c: No bootstrap required, opened a new log
I20260807 19:45:03.552078 13847 ts_tablet_manager.cc:1397] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.001s
W20260807 19:45:03.553668 13813 tablet.cc:2369] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:45:03.560070 13847 raft_consensus.cc:348] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a699e133ce1748a29ef607a294290177" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34267 } } peers { permanent_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 46143 } } peers { permanent_uuid: "9d83c79111ac42c98aed84711a5b3f8c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44527 } }
I20260807 19:45:03.560491 13847 raft_consensus.cc:374] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:03.560606 13847 raft_consensus.cc:729] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d83c79111ac42c98aed84711a5b3f8c, State: Initialized, Role: FOLLOWER
I20260807 19:45:03.561329 13847 consensus_queue.cc:260] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a699e133ce1748a29ef607a294290177" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34267 } } peers { permanent_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 46143 } } peers { permanent_uuid: "9d83c79111ac42c98aed84711a5b3f8c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44527 } }
I20260807 19:45:03.562556 13503 heartbeater.cc:503] Master 127.30.208.62:44429 was elected leader, sending a full tablet report...
I20260807 19:45:03.563115 13847 ts_tablet_manager.cc:1428] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c: Time spent starting tablet: real 0.011s user 0.009s sys 0.002s
W20260807 19:45:03.602288 13665 tablet.cc:2369] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:45:03.634382 13845 raft_consensus.cc:482] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:03.634701 13845 raft_consensus.cc:504] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a699e133ce1748a29ef607a294290177" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34267 } } peers { permanent_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 46143 } } peers { permanent_uuid: "9d83c79111ac42c98aed84711a5b3f8c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44527 } }
I20260807 19:45:03.636431 13845 leader_election.cc:288] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a699e133ce1748a29ef607a294290177 (127.30.208.2:34267), 9d83c79111ac42c98aed84711a5b3f8c (127.30.208.1:44527)
I20260807 19:45:03.643031 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.637084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:52549 (local address 127.30.208.2:34267)
0807 19:45:03.637338 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:03.637343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:03.637360 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:45:03.637444 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:03.637448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:03.637527 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:03.637629 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:03.638154 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.639094 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.641049 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.642207 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.642330 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.642383 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:03.642712 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 19:45:03.642813 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260807 19:45:03.643288 13814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.636789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:34267 (local address 127.30.208.3:52549)
0807 19:45:03.636955 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:03.636972 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:03.637076 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:03.637756 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:03.637761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:03.637797 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:03.638003 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.638015 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.639254 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.639260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.640874 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.640888 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.642365 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.642371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.642551 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.642586 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:45:03.642686 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 19:45:03.642841 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":23}
I20260807 19:45:03.644006 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.637175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44527 (local address 127.30.208.3:40443)
0807 19:45:03.637842 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:03.637856 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:03.637966 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:03.638262 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:03.638266 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:03.638290 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:03.638620 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.638632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.639992 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.639997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.641808 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:03.641823 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.643274 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.643280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:03.643454 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.643486 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:45:03.643577 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 19:45:03.643716 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":2,"thread_start_us":313,"threads_started":1}
I20260807 19:45:03.644301 13832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.637484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:40443 (local address 127.30.208.1:44527)
0807 19:45:03.637686 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:03.637692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:03.637963 (+ 271us) server_negotiation.cc:407] Connection header received
0807 19:45:03.638069 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:03.638074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:03.638153 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:03.638221 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:03.638851 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.639806 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.641955 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.643015 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.643175 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.643256 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:03.643950 (+ 694us) server_negotiation.cc:299] Negotiation successful
0807 19:45:03.644044 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:45:03.644480 13615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c161aea837f04675976fb35b73610534" candidate_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a699e133ce1748a29ef607a294290177" is_pre_election: true
I20260807 19:45:03.645064 13615 raft_consensus.cc:2454] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad242fb73aec4cc49fcdc3f7a5483e5e in term 0.
I20260807 19:45:03.645294 13451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c161aea837f04675976fb35b73610534" candidate_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d83c79111ac42c98aed84711a5b3f8c" is_pre_election: true
I20260807 19:45:03.645828 13451 raft_consensus.cc:2454] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad242fb73aec4cc49fcdc3f7a5483e5e in term 0.
I20260807 19:45:03.645849 13689 leader_election.cc:302] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a699e133ce1748a29ef607a294290177, ad242fb73aec4cc49fcdc3f7a5483e5e; no voters:
I20260807 19:45:03.646273 13845 raft_consensus.cc:2781] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:45:03.646433 13845 raft_consensus.cc:482] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:45:03.646549 13845 raft_consensus.cc:3037] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:03.648772 13845 raft_consensus.cc:504] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a699e133ce1748a29ef607a294290177" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34267 } } peers { permanent_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 46143 } } peers { permanent_uuid: "9d83c79111ac42c98aed84711a5b3f8c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44527 } }
I20260807 19:45:03.649374 13845 leader_election.cc:288] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [CANDIDATE]: Term 1 election: Requested vote from peers a699e133ce1748a29ef607a294290177 (127.30.208.2:34267), 9d83c79111ac42c98aed84711a5b3f8c (127.30.208.1:44527)
I20260807 19:45:03.649812 13615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c161aea837f04675976fb35b73610534" candidate_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a699e133ce1748a29ef607a294290177"
I20260807 19:45:03.649958 13451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c161aea837f04675976fb35b73610534" candidate_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d83c79111ac42c98aed84711a5b3f8c"
I20260807 19:45:03.650045 13615 raft_consensus.cc:3037] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:03.650156 13451 raft_consensus.cc:3037] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:03.653159 13615 raft_consensus.cc:2454] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad242fb73aec4cc49fcdc3f7a5483e5e in term 1.
I20260807 19:45:03.653159 13451 raft_consensus.cc:2454] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad242fb73aec4cc49fcdc3f7a5483e5e in term 1.
I20260807 19:45:03.653565 13689 leader_election.cc:302] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d83c79111ac42c98aed84711a5b3f8c, ad242fb73aec4cc49fcdc3f7a5483e5e; no voters:
I20260807 19:45:03.653865 13845 raft_consensus.cc:2781] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:45:03.654238 13845 raft_consensus.cc:686] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [term 1 LEADER]: Becoming Leader. State: Replica: ad242fb73aec4cc49fcdc3f7a5483e5e, State: Running, Role: LEADER
I20260807 19:45:03.654739 13845 consensus_queue.cc:237] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a699e133ce1748a29ef607a294290177" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34267 } } peers { permanent_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 46143 } } peers { permanent_uuid: "9d83c79111ac42c98aed84711a5b3f8c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44527 } }
I20260807 19:45:03.659139 13280 catalog_manager.cc:5495] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e reported cstate change: term changed from 0 to 1, leader changed from <none> to ad242fb73aec4cc49fcdc3f7a5483e5e (127.30.208.3). New cstate: current_term: 1 leader_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a699e133ce1748a29ef607a294290177" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad242fb73aec4cc49fcdc3f7a5483e5e" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 46143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d83c79111ac42c98aed84711a5b3f8c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44527 } health_report { overall_health: UNKNOWN } } }
W20260807 19:45:03.695544 13504 tablet.cc:2369] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:45:03.702322 13831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.696775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49930 (local address 127.30.208.3:46143)
0807 19:45:03.696987 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:03.696993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:03.697015 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:45:03.697242 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:03.697248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:03.697349 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:03.697471 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:45:03.697942 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.698676 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.700147 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.700542 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.700634 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.701166 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:45:03.701753 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:45:03.701867 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:03.702009 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 19:45:03.702078 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 19:45:03.711094 13615 raft_consensus.cc:1261] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177 [term 1 FOLLOWER]: Refusing update from remote peer ad242fb73aec4cc49fcdc3f7a5483e5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:45:03.711091 13451 raft_consensus.cc:1261] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c [term 1 FOLLOWER]: Refusing update from remote peer ad242fb73aec4cc49fcdc3f7a5483e5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:45:03.712029 13852 consensus_queue.cc:1035] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d83c79111ac42c98aed84711a5b3f8c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 44527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:45:03.712363 13845 consensus_queue.cc:1035] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a699e133ce1748a29ef607a294290177" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:45:03.741953 13280 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:45:03.742249 13280 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40882:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:45:03.745901 13280 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dffcbb35cc764339a85d8fe133c2142f: Sending DeleteTablet for 3 replicas of tablet c161aea837f04675976fb35b73610534
I20260807 19:45:03.747004 13590 tablet_service.cc:1514] Processing DeleteTablet for tablet c161aea837f04675976fb35b73610534 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:03 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:54242
I20260807 19:45:03.747076 13741 tablet_service.cc:1514] Processing DeleteTablet for tablet c161aea837f04675976fb35b73610534 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:03 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:49922
I20260807 19:45:03.747186 13427 tablet_service.cc:1514] Processing DeleteTablet for tablet c161aea837f04675976fb35b73610534 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:03 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:57730
I20260807 19:45:03.748201 13868 tablet_replica.cc:330] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177: stopping tablet replica
I20260807 19:45:03.748688 13868 raft_consensus.cc:2229] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:45:03.749302 13868 raft_consensus.cc:2258] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:03.750111 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 13322
I20260807 19:45:03.751748 13868 ts_tablet_manager.cc:1905] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:45:03.759042 13868 ts_tablet_manager.cc:1918] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:45:03.759344 13868 log.cc:1196] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/c161aea837f04675976fb35b73610534
I20260807 19:45:03.760020 13868 ts_tablet_manager.cc:1939] T c161aea837f04675976fb35b73610534 P a699e133ce1748a29ef607a294290177: Deleting consensus metadata
I20260807 19:45:03.762275 13264 catalog_manager.cc:4841] TS a699e133ce1748a29ef607a294290177 (127.30.208.2:34267): tablet c161aea837f04675976fb35b73610534 (table test-table [id=378fdbac2f6b428aa0310e973a3feefb]) successfully deleted
I20260807 19:45:03.772410 13870 tablet_replica.cc:330] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c: stopping tablet replica
I20260807 19:45:03.772918 13870 raft_consensus.cc:2229] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:45:03.773649 13870 raft_consensus.cc:2258] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:03.774421 13869 tablet_replica.cc:330] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e: stopping tablet replica
I20260807 19:45:03.774784 13832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.751685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57746 (local address 127.30.208.1:44527)
0807 19:45:03.752261 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:03.752267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:03.752287 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:45:03.752346 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:03.752351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:03.752437 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:03.752550 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:03.758029 (+ 5479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.759185 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.761666 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.762413 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.762549 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.766150 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:03.766170 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:03.766190 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:03.766236 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:03.770481 (+ 4245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:03.771520 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:03.771530 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:03.771534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:03.771718 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:03.772225 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:03.772230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:03.772233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:03.772855 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:03.773309 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:03.774099 (+ 790us) server_negotiation.cc:299] Negotiation successful
0807 19:45:03.774333 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450}
I20260807 19:45:03.774861 13869 raft_consensus.cc:2229] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:45:03.775548 13869 raft_consensus.cc:2258] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:03.777814 13870 ts_tablet_manager.cc:1905] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:45:03.778602 13869 ts_tablet_manager.cc:1905] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:45:03.780737 13497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:03.785981 13869 ts_tablet_manager.cc:1918] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:45:03.786158 13869 log.cc:1196] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/c161aea837f04675976fb35b73610534
I20260807 19:45:03.786729 13869 ts_tablet_manager.cc:1939] T c161aea837f04675976fb35b73610534 P ad242fb73aec4cc49fcdc3f7a5483e5e: Deleting consensus metadata
I20260807 19:45:03.788568 13264 catalog_manager.cc:4841] TS ad242fb73aec4cc49fcdc3f7a5483e5e (127.30.208.3:46143): tablet c161aea837f04675976fb35b73610534 (table test-table [id=378fdbac2f6b428aa0310e973a3feefb]) successfully deleted
I20260807 19:45:03.944895 13870 ts_tablet_manager.cc:1918] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:45:03.945125 13870 log.cc:1196] T c161aea837f04675976fb35b73610534 P 9d83c79111ac42c98aed84711a5b3f8c: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/c161aea837f04675976fb35b73610534
I20260807 19:45:03.945478 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 13322
I20260807 19:45:03.963251 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 13507
W20260807 19:45:03.963426 13264 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.208.1:44527 (error 108)
I20260807 19:45:03.965920 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.965410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44527 (local address 127.0.0.1:57760)
0807 19:45:03.965612 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:03.965751 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 19:45:03.966265 13264 catalog_manager.cc:4584] TS 9d83c79111ac42c98aed84711a5b3f8c (127.30.208.1:44527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c161aea837f04675976fb35b73610534: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44527: connect: Connection refused (error 111)
I20260807 19:45:03.977082 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:03.964041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54244 (local address 127.30.208.2:34267)
0807 19:45:03.964221 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:03.964227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:03.964519 (+ 292us) server_negotiation.cc:407] Connection header received
0807 19:45:03.964594 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:03.964600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:03.964700 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:03.964803 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:03.966396 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.967114 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.968357 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:03.968800 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:03.968908 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:03.971417 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:03.971429 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:03.971444 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:03.971478 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:03.974399 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:03.975084 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:03.975091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:03.975093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:03.975224 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:03.975570 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:03.975573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:03.975575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:03.975934 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:03.976212 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:03.976550 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 19:45:03.976758 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:45:03.977792 13659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:04.019877 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:04.019394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44527 (local address 127.0.0.1:57764)
0807 19:45:04.019610 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:04.019699 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260807 19:45:04.020985 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:04.020608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44527 (local address 127.0.0.1:57768)
0807 19:45:04.020756 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:04.020874 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 19:45:04.081748 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:04.081233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44527 (local address 127.0.0.1:57782)
0807 19:45:04.081474 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:04.081570 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:45:04.082861 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:04.082491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44527 (local address 127.0.0.1:57790)
0807 19:45:04.082638 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:04.082736 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 19:45:04.118709 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 13507
I20260807 19:45:04.136541 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 13668
I20260807 19:45:04.153090 13831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:04.137505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49938 (local address 127.30.208.3:46143)
0807 19:45:04.137716 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:04.137722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:04.137747 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:45:04.137907 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:04.137916 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:04.138025 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:04.138151 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:04.139486 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:04.140638 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:04.142018 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:04.142701 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:04.142858 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:04.145633 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:04.145648 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:04.145684 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:04.145744 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:04.149852 (+ 4108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:04.150558 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:04.150569 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:04.150572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:04.150811 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:04.151260 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:04.151265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:04.151269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:04.151775 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:04.152107 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:04.152440 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 19:45:04.152693 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:45:04.153901 13807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:04.163524 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:04.163061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44527 (local address 127.0.0.1:57796)
0807 19:45:04.163269 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:04.163358 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 19:45:04.164582 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:04.164180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44527 (local address 127.0.0.1:57798)
0807 19:45:04.164369 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:04.164443 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 19:45:04.276060 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 13668
I20260807 19:45:04.294915 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 13244
I20260807 19:45:04.296075 13315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:04.407239 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:04.406704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:44527 (local address 127.0.0.1:57806)
0807 19:45:04.406920 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:04.407027 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:44527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 19:45:04.407516 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 13244
I20260807 19:45:04.428910 31552 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 19:45:04.774263 31552 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 19:45:04.774497 31552 mini_ranger.cc:66] Stopping Ranger...
I20260807 19:45:05.119899 31552 mini_ranger.cc:68] Stopped Ranger
2026-08-07 19:45:05.120 UTC [12178] LOG: received smart shutdown request
2026-08-07 19:45:05.122 UTC [12178] LOG: background worker "logical replication launcher" (PID 12189) exited with exit code 1
2026-08-07 19:45:05.123 UTC [12183] LOG: shutting down
2026-08-07 19:45:05.274 UTC [12178] LOG: database system is shut down
2026-08-07T19:45:05Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52308 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:45:05 dist-test-slave-vq2q krb5kdc[13886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:45:05 dist-test-slave-vq2q krb5kdc[13886](info): set up 2 sockets
Aug 07 19:45:05 dist-test-slave-vq2q krb5kdc[13886](info): commencing operation
krb5kdc: starting...
W20260807 19:45:06.471719 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:45:06 dist-test-slave-vq2q krb5kdc[13886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:45:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:45:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:45:06.744441 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:41625
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:41625,127.30.208.61:44057,127.30.208.60:33711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:07.084477 13900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:07.084785 13900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:07.084905 13900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:07.092159 13900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:45:07.092262 13900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:07.092334 13900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:07.092413 13900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:45:07.092473 13900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:45:07.102443 13900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:41625,127.30.208.61:44057,127.30.208.60:33711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:41625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:07.103816 13900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:07.105113 13900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:07.117113 13906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:07.117091 13907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:07.117204 13909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:07.118198 13900 server_base.cc:1034] running on GCE node
I20260807 19:45:07.119016 13900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:07.120471 13900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:07.121723 13900 hybrid_clock.cc:648] HybridClock initialized: now 1786131907121637 us; error 36 us; skew 500 ppm
I20260807 19:45:07.122082 13900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:07 dist-test-slave-vq2q krb5kdc[13886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131907, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:07.125154 13900 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:45:07.126704 13900 webserver.cc:460] Webserver started at http://127.30.208.62:45451/ using document root <none> and password file <none>
I20260807 19:45:07.127393 13900 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:07.127511 13900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:07.127813 13900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:07.130895 13900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "1387cd28d51549f2b3b114dd2ceb64c1"
format_stamp: "Formatted at 2026-08-07 19:45:07 on dist-test-slave-vq2q"
server_key: "b624a7f73f64f7c9d67772fef0afcb8a"
server_key_iv: "adaa03caa016a0e87fe2cd006886ed98"
server_key_version: "encryptionkey@0"
I20260807 19:45:07.131691 13900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "1387cd28d51549f2b3b114dd2ceb64c1"
format_stamp: "Formatted at 2026-08-07 19:45:07 on dist-test-slave-vq2q"
server_key: "b624a7f73f64f7c9d67772fef0afcb8a"
server_key_iv: "adaa03caa016a0e87fe2cd006886ed98"
server_key_version: "encryptionkey@0"
I20260807 19:45:07.136799 13900 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:45:07.140427 13918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:07.141775 13900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 19:45:07.141947 13900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "1387cd28d51549f2b3b114dd2ceb64c1"
format_stamp: "Formatted at 2026-08-07 19:45:07 on dist-test-slave-vq2q"
server_key: "b624a7f73f64f7c9d67772fef0afcb8a"
server_key_iv: "adaa03caa016a0e87fe2cd006886ed98"
server_key_version: "encryptionkey@0"
I20260807 19:45:07.142133 13900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:07.158799 13900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:07.163013 13900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:07.163290 13900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:07.181630 13900 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:41625
I20260807 19:45:07.181667 13974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:41625 every 8 connection(s)
I20260807 19:45:07.183069 13900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:45:07.187026 13975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:07.190505 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 13900
I20260807 19:45:07.190708 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:45:07.191392 31552 external_mini_cluster.cc:1428] Setting key 9c0e8ddd154edde3fc5d58d4da85e1a0
I20260807 19:45:07.193838 13975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:45:07 dist-test-slave-vq2q krb5kdc[13886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131907, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.208.61@KRBTEST.COM; defaulting to no policy
I20260807 19:45:07.218392 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.196770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:41625 (local address 127.0.0.1:32990)
0807 19:45:07.197594 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.197641 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:07.199691 (+ 2050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:07.200654 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:07.200660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:07.201460 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:07.201935 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:07.201960 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.203520 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.203525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:07.205412 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:07.205432 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.206517 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.206524 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:07.206702 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:07.207444 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:07.207474 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:07.210102 (+ 2628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:07.214272 (+ 4170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:07.214284 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:07.214294 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:07.214705 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:07.215205 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:07.215208 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:07.215211 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:07.215369 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:07.216386 (+ 1017us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:07.216391 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:07.216594 (+ 203us) client_negotiation.cc:769] Sending connection context
0807 19:45:07.216802 (+ 208us) client_negotiation.cc:240] Negotiation successful
0807 19:45:07.217274 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":350,"threads_started":1}
I20260807 19:45:07.218420 13978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.198077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32990 (local address 127.30.208.62:41625)
0807 19:45:07.198765 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:07.198773 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:07.198828 (+ 55us) server_negotiation.cc:407] Connection header received
0807 19:45:07.200065 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:07.200073 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:07.200378 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:07.200602 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:07.202139 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.203340 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.205612 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.206238 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.206386 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:07.210395 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:07.210406 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:07.210425 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:07.210464 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:07.214117 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:07.214874 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:07.214885 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:07.214888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:07.215093 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:07.215483 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:07.215488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:07.215491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:07.216238 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:07.216454 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:07.217019 (+ 565us) server_negotiation.cc:299] Negotiation successful
0807 19:45:07.217461 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":117,"server-negotiator.queue_time_us":491,"thread_start_us":318,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.61@KRBTEST.COM" created.
I20260807 19:45:07.221627 13975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:45:07.222798 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.222407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:44057 (local address 127.0.0.1:47068)
0807 19:45:07.222573 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.222684 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 19:45:07.223080 13924 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
I20260807 19:45:07.224653 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.224289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:44057 (local address 127.0.0.1:47084)
0807 19:45:07.224438 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.224520 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 19:45:07.224845 13975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:44057: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.61.keytab.
Entry for principal kudu/127.30.208.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.61.keytab.
I20260807 19:45:07.253284 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.61:44057
--webserver_interface=127.30.208.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.61.keytab
--principal=kudu/127.30.208.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:41625,127.30.208.61:44057,127.30.208.60:33711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:45:07.274222 13975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } attempt: 1
I20260807 19:45:07.275525 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.274977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:44057 (local address 127.0.0.1:47088)
0807 19:45:07.275206 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.275342 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 19:45:07.276556 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.276152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:44057 (local address 127.0.0.1:47092)
0807 19:45:07.276311 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.276399 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 19:45:07.276746 13975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:44057: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
I20260807 19:45:07.345046 13975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } attempt: 2
I20260807 19:45:07.346487 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.345917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:44057 (local address 127.0.0.1:47108)
0807 19:45:07.346179 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.346309 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 19:45:07.347572 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.347150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:44057 (local address 127.0.0.1:47110)
0807 19:45:07.347304 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.347410 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 19:45:07.347775 13975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:44057: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
I20260807 19:45:07.439049 13975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } attempt: 3
I20260807 19:45:07.440379 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.439816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:44057 (local address 127.0.0.1:47114)
0807 19:45:07.440057 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.440179 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 19:45:07.441569 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.441099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:44057 (local address 127.0.0.1:47122)
0807 19:45:07.441290 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.441399 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 19:45:07.441833 13975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:44057: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
I20260807 19:45:07.585121 13975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } attempt: 4
W20260807 19:45:07.585249 13982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:07.585625 13982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:07.585773 13982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 19:45:07.586501 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.585895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:44057 (local address 127.0.0.1:47132)
0807 19:45:07.586223 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.586337 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260807 19:45:07.587491 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.587153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:44057 (local address 127.0.0.1:47140)
0807 19:45:07.587270 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.587363 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 19:45:07.587688 13975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:44057: Network error: Client connection negotiation failed: client connection to 127.30.208.61:44057: connect: Connection refused (error 111)
W20260807 19:45:07.593595 13982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:45:07.593693 13982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:07.593766 13982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:07.593842 13982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:45:07.593899 13982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:45:07.603900 13982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:41625,127.30.208.61:44057,127.30.208.60:33711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.61.keytab
--principal=kudu/127.30.208.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.61:44057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.208.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:07.605260 13982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:07.606585 13982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:07.618216 13988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:07.618211 13989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:07.618160 13991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:07.618628 13982 server_base.cc:1034] running on GCE node
I20260807 19:45:07.619453 13982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:07.620824 13982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:07.622033 13982 hybrid_clock.cc:648] HybridClock initialized: now 1786131907621985 us; error 42 us; skew 500 ppm
I20260807 19:45:07.622366 13982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:07 dist-test-slave-vq2q krb5kdc[13886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131907, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:07.628549 13982 init.cc:377] Logged in from keytab as kudu/127.30.208.61@KRBTEST.COM (short username kudu)
I20260807 19:45:07.630230 13982 webserver.cc:460] Webserver started at http://127.30.208.61:34583/ using document root <none> and password file <none>
I20260807 19:45:07.630931 13982 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:07.631054 13982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:07.631351 13982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:07.634385 13982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/data/instance:
uuid: "0bec3efd09d849de9fc26fb6f0f79ad9"
format_stamp: "Formatted at 2026-08-07 19:45:07 on dist-test-slave-vq2q"
server_key: "1f0b16a3dc83c8b47dd3b8c9109c0707"
server_key_iv: "fb85c51fead59fbabf0aac0644a9dac0"
server_key_version: "encryptionkey@0"
I20260807 19:45:07.635185 13982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/wal/instance:
uuid: "0bec3efd09d849de9fc26fb6f0f79ad9"
format_stamp: "Formatted at 2026-08-07 19:45:07 on dist-test-slave-vq2q"
server_key: "1f0b16a3dc83c8b47dd3b8c9109c0707"
server_key_iv: "fb85c51fead59fbabf0aac0644a9dac0"
server_key_version: "encryptionkey@0"
I20260807 19:45:07.640444 13982 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:45:07.644172 13998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:07.645599 13982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:45:07.645782 13982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/wal
uuid: "0bec3efd09d849de9fc26fb6f0f79ad9"
format_stamp: "Formatted at 2026-08-07 19:45:07 on dist-test-slave-vq2q"
server_key: "1f0b16a3dc83c8b47dd3b8c9109c0707"
server_key_iv: "fb85c51fead59fbabf0aac0644a9dac0"
server_key_version: "encryptionkey@0"
I20260807 19:45:07.646000 13982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:07.665823 13982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:07.669889 13982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:07.670161 13982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:07.688620 13982 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.61:44057
I20260807 19:45:07.688661 14057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.61:44057 every 8 connection(s)
I20260807 19:45:07.690029 13982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/data/info.pb
I20260807 19:45:07.693781 14058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:07.698930 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 13982
I20260807 19:45:07.699143 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-1/wal/instance
I20260807 19:45:07.699683 31552 external_mini_cluster.cc:1428] Setting key 35213c89f6a9e29e57f992e33ab62d2d
I20260807 19:45:07.701153 14058 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:45:07 dist-test-slave-vq2q krb5kdc[13886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131907, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.61@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:07.724159 13978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.704621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32994 (local address 127.30.208.62:41625)
0807 19:45:07.704855 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:07.704862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:07.706170 (+ 1308us) server_negotiation.cc:407] Connection header received
0807 19:45:07.708592 (+ 2422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:07.708599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:07.708767 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:07.708888 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:07.710994 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.712050 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.713749 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.714374 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.714488 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:07.717896 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:07.717902 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:07.717906 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:07.717945 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:07.720924 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:07.721708 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:07.721714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:07.721718 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:07.721883 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:07.722275 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:07.722279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:07.722282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:07.722665 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:07.722818 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:07.723569 (+ 751us) server_negotiation.cc:299] Negotiation successful
0807 19:45:07.723808 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:45:07.724924 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.705086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:41625 (local address 127.0.0.1:32994)
0807 19:45:07.705980 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.706037 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:07.708213 (+ 2176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:07.709277 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:07.709289 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:07.709991 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:07.710743 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:07.710769 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.712250 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.712256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:07.713602 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:07.713616 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.714568 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.714572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:07.714761 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:07.715422 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:07.715446 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:07.717633 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:07.721080 (+ 3447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:07.721087 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:07.721101 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:07.721578 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:07.721980 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:07.721983 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:07.721985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:07.722162 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:07.722786 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:07.722792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:07.723172 (+ 380us) client_negotiation.cc:769] Sending connection context
0807 19:45:07.723410 (+ 238us) client_negotiation.cc:240] Negotiation successful
0807 19:45:07.723795 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":337,"threads_started":1}
WARNING: no policy specified for kudu/127.30.208.60@KRBTEST.COM; defaulting to no policy
I20260807 19:45:07.726783 14058 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.60@KRBTEST.COM" created.
Aug 07 19:45:07 dist-test-slave-vq2q krb5kdc[13886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131907, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.61@KRBTEST.COM for kudu/127.30.208.61@KRBTEST.COM
I20260807 19:45:07.741724 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.727568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:44057 (local address 127.0.0.1:47156)
0807 19:45:07.727718 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.727733 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:07.727819 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:07.728823 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:07.728828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:07.729329 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:07.729489 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:07.729502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.730891 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.730904 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:07.732021 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:07.732037 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.732852 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.732858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:07.733050 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:07.733671 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:07.733694 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:07.736025 (+ 2331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:07.739151 (+ 3126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:07.739156 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:07.739160 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:07.739645 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:07.740018 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:07.740023 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:07.740026 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:07.740210 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:07.740993 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:07.740996 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:07.741114 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 19:45:07.741251 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 19:45:07.741467 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 19:45:07.742043 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.727806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47156 (local address 127.30.208.61:44057)
0807 19:45:07.728420 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:07.728426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:07.728451 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:45:07.728495 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:07.728501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:07.728678 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:07.728809 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:07.729666 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.730739 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.732157 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.732682 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.732784 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:07.736229 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:07.736254 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:07.736267 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:07.736301 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:07.739021 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:07.739784 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:07.739791 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:07.739793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:07.739911 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:07.740375 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:07.740380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:07.740383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:07.740874 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:07.741078 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:07.741408 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 19:45:07.741654 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":337,"threads_started":1}
I20260807 19:45:07.744498 14058 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:45:07.745549 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.745082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40834)
0807 19:45:07.745293 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.745410 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 19:45:07.745814 14001 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
I20260807 19:45:07.747305 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.746988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40840)
0807 19:45:07.747121 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.747209 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 19:45:07.747541 14058 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:33711: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.60.keytab.
Entry for principal kudu/127.30.208.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.60.keytab.
I20260807 19:45:07.762607 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.60:33711
--webserver_interface=127.30.208.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.60.keytab
--principal=kudu/127.30.208.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:41625,127.30.208.61:44057,127.30.208.60:33711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:45:07.796888 14058 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } attempt: 1
I20260807 19:45:07.798139 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.797645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40846)
0807 19:45:07.797880 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.797988 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260807 19:45:07.799237 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.798864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40858)
0807 19:45:07.799035 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.799110 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 19:45:07.799463 14058 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:33711: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
I20260807 19:45:07.867689 14058 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } attempt: 2
I20260807 19:45:07.868811 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.868361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40862)
0807 19:45:07.868566 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.868657 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 19:45:07.869939 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.869563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40864)
0807 19:45:07.869725 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.869818 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 19:45:07.870146 14058 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:33711: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
I20260807 19:45:07.886915 13975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } attempt: 5
Aug 07 19:45:07 dist-test-slave-vq2q krb5kdc[13886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131907, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for kudu/127.30.208.61@KRBTEST.COM
I20260807 19:45:07.900185 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.887802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47168 (local address 127.30.208.61:44057)
0807 19:45:07.888039 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:07.888045 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:07.888067 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:45:07.888132 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:07.888138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:07.888223 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:07.888306 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:07.889137 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.889867 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.891047 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.891506 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.891618 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:07.895059 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:07.895064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:07.895067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:07.895102 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:07.897648 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:07.898281 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:07.898287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:07.898289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:07.898412 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:07.898777 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:07.898782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:07.898786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:07.899082 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:07.899220 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:07.899637 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 19:45:07.899848 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260807 19:45:07.900185 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.887677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:44057 (local address 127.0.0.1:47168)
0807 19:45:07.887865 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.887884 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:07.888019 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:07.888372 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:07.888377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:07.888778 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:07.888961 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:07.888971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.890017 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.890021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:07.890899 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:07.890912 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:07.891654 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:07.891660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:07.891826 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:07.892462 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:07.892480 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:07.894850 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:07.897789 (+ 2939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:07.897795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:07.897800 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:07.898166 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:07.898520 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:07.898526 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:07.898530 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:07.898674 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:07.899201 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:07.899206 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:07.899362 (+ 156us) client_negotiation.cc:769] Sending connection context
0807 19:45:07.899517 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 19:45:07.899815 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 19:45:07.901322 13975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:45:07.902422 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.902014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40874)
0807 19:45:07.902190 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.902269 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 19:45:07.903273 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.902956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40878)
0807 19:45:07.903088 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.903160 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 19:45:07.903478 13975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:33711: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
I20260807 19:45:07.954715 13975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } attempt: 1
I20260807 19:45:07.955855 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.955385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40894)
0807 19:45:07.955589 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.955684 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 19:45:07.956979 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.956587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40906)
0807 19:45:07.956764 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.956866 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 19:45:07.957233 13975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:33711: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
I20260807 19:45:07.961335 14058 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } attempt: 3
I20260807 19:45:07.962502 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.962053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40920)
0807 19:45:07.962252 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.962342 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 19:45:07.963616 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:07.963183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40928)
0807 19:45:07.963421 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:07.963501 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260807 19:45:07.963836 14058 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:33711: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
I20260807 19:45:08.025446 13975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } attempt: 2
I20260807 19:45:08.026733 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.026189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40932)
0807 19:45:08.026463 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:08.026565 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 19:45:08.027838 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.027466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40940)
0807 19:45:08.027588 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:08.027659 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 19:45:08.028035 13975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:33711: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
W20260807 19:45:08.098272 14065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:08.098623 14065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:08.098732 14065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:08.105839 14065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:45:08.105942 14065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:08.105995 14065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:08.106051 14065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:45:08.106094 14065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:45:08.107031 14058 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } attempt: 4
I20260807 19:45:08.108215 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.107701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40952)
0807 19:45:08.107957 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:08.108048 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 19:45:08.109395 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.108965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40956)
0807 19:45:08.109134 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:08.109254 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 19:45:08.109637 14058 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:33711: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
I20260807 19:45:08.116411 14065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:41625,127.30.208.61:44057,127.30.208.60:33711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.60.keytab
--principal=kudu/127.30.208.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.60:33711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.208.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:08.117692 14065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:08.118969 14065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:08.130106 14070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:08.130125 14071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:08.130147 14074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:08.131175 14065 server_base.cc:1034] running on GCE node
I20260807 19:45:08.132134 14065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:08.133694 14065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:08.134217 13975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } attempt: 3
I20260807 19:45:08.134905 14065 hybrid_clock.cc:648] HybridClock initialized: now 1786131908134842 us; error 51 us; skew 500 ppm
I20260807 19:45:08.135207 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.134752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40962)
0807 19:45:08.134937 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:08.135033 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 19:45:08.135234 14065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:45:08.136174 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.135797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40978)
0807 19:45:08.135990 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:08.136064 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260807 19:45:08.136384 13975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:33711: Network error: Client connection negotiation failed: client connection to 127.30.208.60:33711: connect: Connection refused (error 111)
Aug 07 19:45:08 dist-test-slave-vq2q krb5kdc[13886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:08.138257 14065 init.cc:377] Logged in from keytab as kudu/127.30.208.60@KRBTEST.COM (short username kudu)
I20260807 19:45:08.139642 14065 webserver.cc:460] Webserver started at http://127.30.208.60:44085/ using document root <none> and password file <none>
I20260807 19:45:08.140317 14065 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:08.140434 14065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:08.140725 14065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:08.143675 14065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/data/instance:
uuid: "aecd503bbfe44b7787de2be2f1c5c8bf"
format_stamp: "Formatted at 2026-08-07 19:45:08 on dist-test-slave-vq2q"
server_key: "e08ecd3fc53d95515d1aa52700f9b902"
server_key_iv: "126cfd818e4f7308efde39b2d4090f49"
server_key_version: "encryptionkey@0"
I20260807 19:45:08.144474 14065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/wal/instance:
uuid: "aecd503bbfe44b7787de2be2f1c5c8bf"
format_stamp: "Formatted at 2026-08-07 19:45:08 on dist-test-slave-vq2q"
server_key: "e08ecd3fc53d95515d1aa52700f9b902"
server_key_iv: "126cfd818e4f7308efde39b2d4090f49"
server_key_version: "encryptionkey@0"
I20260807 19:45:08.149621 14065 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:45:08.152926 14081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:08.154311 14065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:45:08.154476 14065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/wal
uuid: "aecd503bbfe44b7787de2be2f1c5c8bf"
format_stamp: "Formatted at 2026-08-07 19:45:08 on dist-test-slave-vq2q"
server_key: "e08ecd3fc53d95515d1aa52700f9b902"
server_key_iv: "126cfd818e4f7308efde39b2d4090f49"
server_key_version: "encryptionkey@0"
I20260807 19:45:08.154660 14065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:08.184531 14065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:08.189893 14065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:08.190198 14065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:08.207998 14065 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.60:33711
I20260807 19:45:08.208053 14135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.60:33711 every 8 connection(s)
I20260807 19:45:08.209352 14065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/data/info.pb
I20260807 19:45:08.210597 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 14065
I20260807 19:45:08.210767 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786131702042267-31552-0/minicluster-data/master-2/wal/instance
I20260807 19:45:08.211242 31552 external_mini_cluster.cc:1428] Setting key caa4e715ef17bf7b77308f0d2ad39328
I20260807 19:45:08.213677 14137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:08.221796 14137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:45:08 dist-test-slave-vq2q krb5kdc[13886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.60@KRBTEST.COM
Aug 07 19:45:08 dist-test-slave-vq2q krb5kdc[13886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.60@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:08.235455 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.213224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40982 (local address 127.30.208.60:33711)
0807 19:45:08.214063 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:08.214074 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:08.214114 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:45:08.215737 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:08.215758 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:08.216135 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:08.216466 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:08.218342 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.219710 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.221594 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.222314 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.222465 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.227561 (+ 5096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:08.227569 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:08.227578 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:08.227626 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:08.231083 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.231901 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.231910 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.231918 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.232209 (+ 291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.232573 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.232578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.232581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.233327 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:08.233565 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:08.234046 (+ 481us) server_negotiation.cc:299] Negotiation successful
0807 19:45:08.234470 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"spinlock_wait_cycles":17792,"thread_start_us":396,"threads_started":1}
I20260807 19:45:08.241264 14141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.224793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:41625 (local address 127.0.0.1:32998)
0807 19:45:08.225596 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:08.225644 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:08.225837 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:08.226572 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:08.226581 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:08.227390 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:08.227753 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.227764 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.228891 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.228895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.230445 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.230462 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.231245 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.231249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.231368 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.231955 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:08.231977 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:08.234631 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:08.238635 (+ 4004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:08.238646 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:08.238652 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:08.239090 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:08.239487 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:08.239491 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:08.239493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:08.239632 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:08.240168 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:08.240176 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:08.240345 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 19:45:08.240499 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 19:45:08.240821 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":399,"threads_started":1}
I20260807 19:45:08.241330 14142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.224929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32998 (local address 127.30.208.62:41625)
0807 19:45:08.226230 (+ 1301us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:08.226235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:08.226249 (+ 14us) server_negotiation.cc:407] Connection header received
0807 19:45:08.226297 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:08.226301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:08.226369 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:08.226474 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:08.227908 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.228728 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.230603 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.231070 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.231176 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.234854 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:08.234860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:08.234863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:08.234897 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:08.238466 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.239222 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.239226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.239228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.239368 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.239746 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.239750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.239752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.240036 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:08.240196 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:08.240648 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 19:45:08.240866 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1200,"thread_start_us":1085,"threads_started":1}
I20260807 19:45:08.243180 14137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:45:08 dist-test-slave-vq2q krb5kdc[13886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
Aug 07 19:45:08 dist-test-slave-vq2q krb5kdc[13886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.60@KRBTEST.COM for kudu/127.30.208.61@KRBTEST.COM
I20260807 19:45:08.256096 14143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.238471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.30.208.62:41625)
0807 19:45:08.241661 (+ 3190us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:08.241665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:08.241684 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:45:08.241731 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:08.241734 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:08.241805 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:08.241908 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:08.243375 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.244260 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.245579 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.246026 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.246137 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.250412 (+ 4275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:08.250420 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:08.250425 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:08.250466 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:08.253122 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.253988 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.253994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.253997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.254163 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.254546 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.254551 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.254554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.254880 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:08.255171 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:08.255438 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 19:45:08.255629 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3075,"thread_start_us":2914,"threads_started":1}
I20260807 19:45:08.256593 14141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.243971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:44057 (local address 127.0.0.1:47184)
0807 19:45:08.244251 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:08.244265 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:08.244362 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:08.244664 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:08.244669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:08.245021 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:08.245148 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.245155 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.246259 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.246264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.247311 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.247324 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.248023 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.248030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.248210 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.248673 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:08.248691 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:08.250876 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:08.254076 (+ 3200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:08.254081 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:08.254084 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:08.254545 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:08.254897 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:08.254901 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:08.254905 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:08.255079 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:08.255654 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:08.255659 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:08.255848 (+ 189us) client_negotiation.cc:769] Sending connection context
0807 19:45:08.256006 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 19:45:08.256294 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201}
I20260807 19:45:08.256631 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.243965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47184 (local address 127.30.208.61:44057)
0807 19:45:08.244150 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:08.244154 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:08.244333 (+ 179us) server_negotiation.cc:407] Connection header received
0807 19:45:08.244481 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:08.244488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:08.244549 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:08.244630 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:08.245325 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.246128 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.247444 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.247871 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.247989 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.251059 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:08.251064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:08.251067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:08.251101 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:08.253936 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.254665 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.254669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.254671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.254793 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.255200 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.255205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.255208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.255526 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:08.255688 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:08.256119 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 19:45:08.256324 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:45:08.257810 14137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:45:08 dist-test-slave-vq2q krb5kdc[13886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.60@KRBTEST.COM for kudu/127.30.208.60@KRBTEST.COM
Aug 07 19:45:08 dist-test-slave-vq2q krb5kdc[13886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.61@KRBTEST.COM
I20260807 19:45:08.270632 14141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.258539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:40988)
0807 19:45:08.258716 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:08.258733 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:08.258826 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:08.259149 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:08.259152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:08.259492 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:08.259626 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.259634 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.260702 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.260708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.261719 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.261734 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.262563 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.262569 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.262756 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.263329 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:08.263353 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:08.265316 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:08.268210 (+ 2894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:08.268215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:08.268218 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:08.268690 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:08.269084 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:08.269089 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:08.269092 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:08.269280 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:08.269788 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:08.269793 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:08.269959 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 19:45:08.270078 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 19:45:08.270294 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 19:45:08.270888 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.258738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40988 (local address 127.30.208.60:33711)
0807 19:45:08.258898 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:08.258903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:08.258921 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:45:08.258967 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:08.258972 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:08.259038 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:08.259135 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:08.259823 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.260566 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.261856 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.262411 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.262501 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.265489 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:08.265496 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:08.265499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:08.265527 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:08.268080 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.268824 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.268828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.268830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.268965 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.269386 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.269389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.269391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.269672 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:08.269782 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:08.270226 (+ 444us) server_negotiation.cc:299] Negotiation successful
0807 19:45:08.270485 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:45:08.272782 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.257913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47200 (local address 127.30.208.61:44057)
0807 19:45:08.258064 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:08.258070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:08.258108 (+ 38us) server_negotiation.cc:407] Connection header received
0807 19:45:08.258285 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:08.258288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:08.258356 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:08.258441 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:08.260037 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.260838 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.262210 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.262680 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.262809 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.266989 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:08.266995 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:08.267000 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:08.267041 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:08.269816 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.270633 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.270639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.270643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.270815 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.271307 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.271333 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.271336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.271630 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:08.271855 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:08.272201 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 19:45:08.272430 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:45:08.276772 14137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf: Bootstrap starting.
I20260807 19:45:08.277681 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:45:08.280747 14137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:08.282159 14137 log.cc:824] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:08.285514 14137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf: No bootstrap required, opened a new log
I20260807 19:45:08.297122 14137 raft_consensus.cc:348] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } }
I20260807 19:45:08.297637 14137 raft_consensus.cc:374] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:08.297856 14137 raft_consensus.cc:729] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aecd503bbfe44b7787de2be2f1c5c8bf, State: Initialized, Role: FOLLOWER
I20260807 19:45:08.298735 14137 consensus_queue.cc:260] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } }
I20260807 19:45:08.302798 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.283682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.30.208.61:44057)
0807 19:45:08.283876 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:08.283883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:08.285753 (+ 1870us) server_negotiation.cc:407] Connection header received
0807 19:45:08.286035 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:08.286041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:08.286122 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:08.286189 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:08.288543 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.289655 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.291645 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.292253 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.292348 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.295742 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:08.295748 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:08.295752 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:08.295795 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:08.299475 (+ 3680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.300401 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.300407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.300409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.300566 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.300971 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.300976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.300979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.301410 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:08.301621 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:08.302194 (+ 573us) server_negotiation.cc:299] Negotiation successful
0807 19:45:08.302445 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:45:08.302798 14143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.283322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.30.208.62:41625)
0807 19:45:08.283527 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:08.283533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:08.284440 (+ 907us) server_negotiation.cc:407] Connection header received
0807 19:45:08.284598 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:08.284604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:08.284687 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:08.284772 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:08.287048 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.288109 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.290358 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.290996 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.291093 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.295798 (+ 4705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:08.295805 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:08.295809 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:08.295851 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:08.299288 (+ 3437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.300303 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.300309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.300312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.300468 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.300975 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.300980 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.300983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.301386 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:08.301622 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:08.302197 (+ 575us) server_negotiation.cc:299] Negotiation successful
0807 19:45:08.302444 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:45:08.306159 14153 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } } }
I20260807 19:45:08.306620 14153 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:45:08.307516 14137 sys_catalog.cc:564] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:45:08.311497 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.286254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40998 (local address 127.30.208.60:33711)
0807 19:45:08.286454 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:08.286459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:08.295558 (+ 9099us) server_negotiation.cc:407] Connection header received
0807 19:45:08.295691 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:08.295695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:08.295780 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:08.295885 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:08.297617 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.298625 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.300789 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.301600 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.301706 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.305531 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:08.305536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:08.305540 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:08.305574 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:08.308564 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.309535 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.309540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.309544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.309719 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.310086 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.310091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.310095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.310379 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:08.310633 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:08.310928 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 19:45:08.311159 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:45:08.313591 13975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } attempt: 4
W20260807 19:45:08.317907 14169 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:45:08.318135 14169 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 07 19:45:08 dist-test-slave-vq2q krb5kdc[13886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131907, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for kudu/127.30.208.60@KRBTEST.COM
I20260807 19:45:08.314769 14065 master_runner.cc:186] Error getting master registration for 127.30.208.62:41625: OK, instance_id { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" instance_seqno: 1786131907175877 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:45:08.325682 14065 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:45:08.331900 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.314175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:41014)
0807 19:45:08.314341 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:08.314360 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:08.314465 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:08.314972 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:08.314977 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:08.315448 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:08.315649 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.315663 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.317386 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.317392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.318764 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.318781 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.319939 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.319945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.320117 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.320814 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:08.320839 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:08.325439 (+ 4600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:08.329324 (+ 3885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:08.329329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:08.329333 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:08.329900 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:08.330337 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:08.330341 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:08.330343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:08.330488 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:08.331135 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:08.331139 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:08.331289 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 19:45:08.331407 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 19:45:08.331622 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 19:45:08.332178 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.314210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41014 (local address 127.30.208.60:33711)
0807 19:45:08.314365 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:08.314370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:08.314409 (+ 39us) server_negotiation.cc:407] Connection header received
0807 19:45:08.314582 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:08.314587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:08.314680 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:08.314780 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:08.315826 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.317201 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.318955 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.319591 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.319730 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.325674 (+ 5944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:08.325681 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:08.325686 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:08.325731 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:08.329101 (+ 3370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.330003 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.330009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.330013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.330196 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.330604 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.330610 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.330613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.331031 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:08.331188 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:08.331553 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 19:45:08.331803 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 19:45:08.336444 13975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1: Bootstrap starting.
I20260807 19:45:08.340544 13975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:08.341605 13975 log.cc:824] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:08.344208 13975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1: No bootstrap required, opened a new log
I20260807 19:45:08.352403 13975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } }
I20260807 19:45:08.352705 13975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:08.352816 13975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1387cd28d51549f2b3b114dd2ceb64c1, State: Initialized, Role: FOLLOWER
I20260807 19:45:08.353412 13975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } }
I20260807 19:45:08.354300 14172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } } }
I20260807 19:45:08.354713 14172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:45:08.355474 13975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:45:08.364125 14153 raft_consensus.cc:482] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:08.364451 14153 raft_consensus.cc:504] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } }
I20260807 19:45:08.366153 14153 leader_election.cc:288] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1387cd28d51549f2b3b114dd2ceb64c1 (127.30.208.62:41625), 0bec3efd09d849de9fc26fb6f0f79ad9 (127.30.208.61:44057)
I20260807 19:45:08.367445 13949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" is_pre_election: true
I20260807 19:45:08.368139 13949 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aecd503bbfe44b7787de2be2f1c5c8bf in term 0.
I20260807 19:45:08.369131 14084 leader_election.cc:302] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1387cd28d51549f2b3b114dd2ceb64c1, aecd503bbfe44b7787de2be2f1c5c8bf; no voters:
I20260807 19:45:08.370362 14153 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:45:08.370515 14153 raft_consensus.cc:482] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:45:08.370898 14153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:08.374306 14153 raft_consensus.cc:504] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } }
W20260807 19:45:08.376242 14186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:45:08.376372 14186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:45:08.377552 13949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1387cd28d51549f2b3b114dd2ceb64c1"
I20260807 19:45:08.377796 13949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:08.366061 13900 master_runner.cc:186] Error getting master registration for 127.30.208.61:44057: OK, instance_id { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" instance_seqno: 1786131907683348 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:45:08.380728 13900 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:45:08.381587 13949 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aecd503bbfe44b7787de2be2f1c5c8bf in term 1.
I20260807 19:45:08.382164 14084 leader_election.cc:302] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1387cd28d51549f2b3b114dd2ceb64c1, aecd503bbfe44b7787de2be2f1c5c8bf; no voters:
I20260807 19:45:08.382229 14153 leader_election.cc:288] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [CANDIDATE]: Term 1 election: Requested vote from peers 1387cd28d51549f2b3b114dd2ceb64c1 (127.30.208.62:41625), 0bec3efd09d849de9fc26fb6f0f79ad9 (127.30.208.61:44057)
I20260807 19:45:08.382535 14153 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:45:08.382678 14153 raft_consensus.cc:686] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [term 1 LEADER]: Becoming Leader. State: Replica: aecd503bbfe44b7787de2be2f1c5c8bf, State: Running, Role: LEADER
I20260807 19:45:08.383834 14153 consensus_queue.cc:237] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } }
I20260807 19:45:08.402818 14188 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader aecd503bbfe44b7787de2be2f1c5c8bf. Latest consensus state: current_term: 1 leader_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } } }
I20260807 19:45:08.403479 14188 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:08.408847 14058 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } attempt: 5
I20260807 19:45:08.409825 14190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:45:08.413277 14190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 19:45:08.416090 13926 tablet.cc:2369] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 19:45:08 dist-test-slave-vq2q krb5kdc[13886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131907, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.61@KRBTEST.COM for kudu/127.30.208.60@KRBTEST.COM
I20260807 19:45:08.430519 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.410073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41018 (local address 127.30.208.60:33711)
0807 19:45:08.410257 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:08.410263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:08.410287 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:45:08.410355 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:08.410360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:08.410441 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:08.410560 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:08.411646 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.412831 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.414749 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.415477 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.415600 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.419807 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:08.419814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:08.419819 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:08.419872 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:08.423953 (+ 4081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.425514 (+ 1561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.425522 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.425526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.425881 (+ 355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:08.426769 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:08.426777 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:08.426780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:08.427632 (+ 852us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:08.427955 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:08.429008 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0807 19:45:08.429635 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:45:08.432269 14060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:08.409619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:33711 (local address 127.0.0.1:41018)
0807 19:45:08.409864 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:08.409888 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:08.410025 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:08.410654 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:08.410659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:08.411234 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:08.411455 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.411470 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.413036 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.413042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.414561 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:08.414580 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:08.415808 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:08.415815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:08.416086 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:08.416784 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:08.416809 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:08.419595 (+ 2786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:08.424146 (+ 4551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:08.424153 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:08.424157 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:08.425246 (+ 1089us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:08.426175 (+ 929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:08.426209 (+ 34us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:08.426213 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:08.426510 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:08.428099 (+ 1589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:08.428104 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:08.428384 (+ 280us) client_negotiation.cc:769] Sending connection context
0807 19:45:08.430950 (+ 2566us) client_negotiation.cc:240] Negotiation successful
0807 19:45:08.431552 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 19:45:08.438804 14058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9: Bootstrap starting.
W20260807 19:45:08.442497 14087 tablet.cc:2369] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:45:08.443131 14058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:08.445221 14058 log.cc:824] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:08.446640 13949 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [term 1 FOLLOWER]: Refusing update from remote peer aecd503bbfe44b7787de2be2f1c5c8bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:45:08.447764 14153 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:45:08.457998 14058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9: No bootstrap required, opened a new log
I20260807 19:45:08.472470 14172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aecd503bbfe44b7787de2be2f1c5c8bf. Latest consensus state: current_term: 1 leader_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } } }
I20260807 19:45:08.472848 14172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:45:08.473655 14058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } }
I20260807 19:45:08.474093 14058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:08.474216 14058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bec3efd09d849de9fc26fb6f0f79ad9, State: Initialized, Role: FOLLOWER
I20260807 19:45:08.474865 14058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } }
I20260807 19:45:08.478348 14191 mvcc.cc:205] Tried to move back new op lower bound from 7315996296984932352 to 7315996296749264896. Current Snapshot: MvccSnapshot[applied={T|T < 7315996296984932352}]
I20260807 19:45:08.477771 14195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } } }
I20260807 19:45:08.485517 14058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:45:08.485582 14030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" is_pre_election: true
I20260807 19:45:08.486265 14030 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aecd503bbfe44b7787de2be2f1c5c8bf in term 0.
I20260807 19:45:08.487540 14028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:08.490998 14029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9"
I20260807 19:45:08.493469 14028 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9 [term 1 FOLLOWER]: Refusing update from remote peer aecd503bbfe44b7787de2be2f1c5c8bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:45:08.494485 14188 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:45:08.505162 14195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:45:08.514532 14172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } } }
I20260807 19:45:08.514798 14172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:45:08.515602 14188 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } } }
I20260807 19:45:08.515897 14188 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:08.522382 14190 catalog_manager.cc:1347] Generated new cluster ID: 5bf3853d404e47a2aa5d5b6d74059f1c
I20260807 19:45:08.522514 14190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:45:08.550293 14205 mvcc.cc:205] Tried to move back new op lower bound from 7315996296984932352 to 7315996296749264896. Current Snapshot: MvccSnapshot[applied={T|T < 7315996296984932352}]
I20260807 19:45:08.558615 14195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aecd503bbfe44b7787de2be2f1c5c8bf. Latest consensus state: current_term: 1 leader_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } } }
I20260807 19:45:08.558983 14195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:45:08.559695 14200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } } }
I20260807 19:45:08.560524 14200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:45:08.587672 14188 sys_catalog.cc:455] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1387cd28d51549f2b3b114dd2ceb64c1" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41625 } } peers { permanent_uuid: "0bec3efd09d849de9fc26fb6f0f79ad9" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 44057 } } peers { permanent_uuid: "aecd503bbfe44b7787de2be2f1c5c8bf" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 33711 } } }
I20260807 19:45:08.588023 14188 sys_catalog.cc:458] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:08.597409 14190 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:45:08.599304 14190 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:45:08.627763 14190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aecd503bbfe44b7787de2be2f1c5c8bf: Generated new TSK 0
I20260807 19:45:08.629035 14190 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:45:08.633816 14218 catalog_manager.cc:1259] Loaded cluster ID: 5bf3853d404e47a2aa5d5b6d74059f1c
I20260807 19:45:08.633930 14218 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9: loading cluster ID for follower catalog manager: success
I20260807 19:45:08.637640 14218 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9: acquiring CA information for follower catalog manager: success
I20260807 19:45:08.640348 14218 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0bec3efd09d849de9fc26fb6f0f79ad9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:45:09.378484 14186 catalog_manager.cc:1259] Loaded cluster ID: 5bf3853d404e47a2aa5d5b6d74059f1c
I20260807 19:45:09.378600 14186 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1: loading cluster ID for follower catalog manager: success
I20260807 19:45:09.381397 14186 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1: acquiring CA information for follower catalog manager: success
I20260807 19:45:09.383389 14186 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1387cd28d51549f2b3b114dd2ceb64c1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:45:09.770763 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 13900
I20260807 19:45:09.772245 13969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:09.892146 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 13900
I20260807 19:45:09.914507 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 13982
I20260807 19:45:09.915684 14052 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:10.025455 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 13982
I20260807 19:45:10.046281 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 14065
I20260807 19:45:10.047441 14129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:10.160724 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 14065
2026-08-07T19:45:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4773 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:45:10 dist-test-slave-vq2q krb5kdc[14229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:45:10 dist-test-slave-vq2q krb5kdc[14229](info): set up 2 sockets
Aug 07 19:45:10 dist-test-slave-vq2q krb5kdc[14229](info): commencing operation
krb5kdc: starting...
W20260807 19:45:11.264537 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:45:11 dist-test-slave-vq2q krb5kdc[14229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:45:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:45:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:45:11.553719 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33689
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:37125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:33689
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:11.923506 14243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:11.923863 14243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:11.924014 14243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:11.931998 14243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:45:11.932156 14243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:11.932232 14243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:11.932328 14243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:45:11.932400 14243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:45:11.942660 14243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:33689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:11.944046 14243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:11.945473 14243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:11.957701 14250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:11.957581 14251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:11.957598 14253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:11.958305 14243 server_base.cc:1034] running on GCE node
I20260807 19:45:11.959344 14243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:11.961340 14243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:11.962615 14243 hybrid_clock.cc:648] HybridClock initialized: now 1786131911962547 us; error 68 us; skew 500 ppm
I20260807 19:45:11.963047 14243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:11 dist-test-slave-vq2q krb5kdc[14229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:11.966460 14243 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:45:11.968061 14243 webserver.cc:460] Webserver started at http://127.30.208.62:34477/ using document root <none> and password file <none>
I20260807 19:45:11.968762 14243 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:11.968892 14243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:11.969218 14243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:11.972460 14243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "e6ba5b52ae3d44acabde6d8e67f70b81"
format_stamp: "Formatted at 2026-08-07 19:45:11 on dist-test-slave-vq2q"
server_key: "915d5066406290f1b2f6514f159d18db"
server_key_iv: "dc8a3c5da9f061a8e65b019c8df43192"
server_key_version: "encryptionkey@0"
I20260807 19:45:11.973302 14243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "e6ba5b52ae3d44acabde6d8e67f70b81"
format_stamp: "Formatted at 2026-08-07 19:45:11 on dist-test-slave-vq2q"
server_key: "915d5066406290f1b2f6514f159d18db"
server_key_iv: "dc8a3c5da9f061a8e65b019c8df43192"
server_key_version: "encryptionkey@0"
I20260807 19:45:11.978890 14243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:45:11.982599 14260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:11.984027 14243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:45:11.984216 14243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "e6ba5b52ae3d44acabde6d8e67f70b81"
format_stamp: "Formatted at 2026-08-07 19:45:11 on dist-test-slave-vq2q"
server_key: "915d5066406290f1b2f6514f159d18db"
server_key_iv: "dc8a3c5da9f061a8e65b019c8df43192"
server_key_version: "encryptionkey@0"
I20260807 19:45:11.984424 14243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:12.001889 14243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:12.006517 14243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:12.006829 14243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:12.025868 14243 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:33689
I20260807 19:45:12.025903 14320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:33689 every 8 connection(s)
I20260807 19:45:12.027295 14243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:45:12.031255 14321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:12.031965 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 14243
I20260807 19:45:12.032232 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:45:12.032908 31552 external_mini_cluster.cc:1428] Setting key bb777a4c6a48badb98dc7b653fb732f1
I20260807 19:45:12.046278 14321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81: Bootstrap starting.
Aug 07 19:45:12 dist-test-slave-vq2q krb5kdc[14229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:12.050685 14321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:12.052119 14321 log.cc:824] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:12.055367 14321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81: No bootstrap required, opened a new log
I20260807 19:45:12.060294 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:12.034909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45462 (local address 127.30.208.62:33689)
0807 19:45:12.035772 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:12.035791 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:12.035846 (+ 55us) server_negotiation.cc:407] Connection header received
0807 19:45:12.037910 (+ 2064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:12.037943 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:12.038533 (+ 590us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:12.039179 (+ 646us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:12.040687 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:12.042726 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:12.044203 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:12.045351 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:12.045549 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:12.052057 (+ 6508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:12.052067 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:12.052083 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:12.052134 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:12.055272 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:12.056398 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:12.056417 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:12.056424 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:12.056618 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:12.057102 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:12.057106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:12.057108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:12.057786 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:12.058065 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:12.058589 (+ 524us) server_negotiation.cc:299] Negotiation successful
0807 19:45:12.059076 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":359,"threads_started":1}
I20260807 19:45:12.065466 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:45:12.065650 14321 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ba5b52ae3d44acabde6d8e67f70b81" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33689 } }
I20260807 19:45:12.066085 14321 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:12.066177 14321 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ba5b52ae3d44acabde6d8e67f70b81, State: Initialized, Role: FOLLOWER
I20260807 19:45:12.066777 14321 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ba5b52ae3d44acabde6d8e67f70b81" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33689 } }
I20260807 19:45:12.067011 14321 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:45:12.067139 14321 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:45:12.067301 14321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:12.069940 14321 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ba5b52ae3d44acabde6d8e67f70b81" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33689 } }
I20260807 19:45:12.070451 14321 leader_election.cc:302] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6ba5b52ae3d44acabde6d8e67f70b81; no voters:
I20260807 19:45:12.071059 14321 leader_election.cc:288] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:45:12.071206 14327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:45:12.071915 14327 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [term 1 LEADER]: Becoming Leader. State: Replica: e6ba5b52ae3d44acabde6d8e67f70b81, State: Running, Role: LEADER
I20260807 19:45:12.072451 14327 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ba5b52ae3d44acabde6d8e67f70b81" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33689 } }
I20260807 19:45:12.073129 14321 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:45:12.076375 14328 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6ba5b52ae3d44acabde6d8e67f70b81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ba5b52ae3d44acabde6d8e67f70b81" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33689 } } }
I20260807 19:45:12.076721 14328 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:12.077062 14329 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6ba5b52ae3d44acabde6d8e67f70b81. Latest consensus state: current_term: 1 leader_uuid: "e6ba5b52ae3d44acabde6d8e67f70b81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ba5b52ae3d44acabde6d8e67f70b81" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33689 } } }
I20260807 19:45:12.077409 14329 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81 [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:12.077673 14338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:45:12.084144 14338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:45:12.100239 14338 catalog_manager.cc:1347] Generated new cluster ID: f9e5678eb82342e39d1302accee29e6b
I20260807 19:45:12.100405 14338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:45:12.125560 14338 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:45:12.127022 14338 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:45:12.139931 14338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6ba5b52ae3d44acabde6d8e67f70b81: Generated new TSK 0
I20260807 19:45:12.140868 14338 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:45:12.178190 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 14243
I20260807 19:45:12.179486 14315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:12.279493 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 14243
2026-08-07T19:45:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2115 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:45:12 dist-test-slave-vq2q krb5kdc[14354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:45:12 dist-test-slave-vq2q krb5kdc[14354](info): set up 2 sockets
Aug 07 19:45:12 dist-test-slave-vq2q krb5kdc[14354](info): commencing operation
krb5kdc: starting...
W20260807 19:45:13.379587 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.006s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:45:13 dist-test-slave-vq2q krb5kdc[14354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:45:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:45:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:45:13.666808 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33657
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:33657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:14.028565 14370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:14.028925 14370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:14.029055 14370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:14.036679 14370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:45:14.036837 14370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:14.036924 14370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:14.037007 14370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:45:14.037068 14370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:45:14.047936 14370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:33657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:14.049401 14370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:14.050738 14370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:14.062392 14378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:14.062530 14375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:14.062554 14376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:14.063550 14370 server_base.cc:1034] running on GCE node
I20260807 19:45:14.064460 14370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:14.066020 14370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:14.067230 14370 hybrid_clock.cc:648] HybridClock initialized: now 1786131914067179 us; error 44 us; skew 500 ppm
I20260807 19:45:14.067579 14370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:14 dist-test-slave-vq2q krb5kdc[14354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131914, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:14.070685 14370 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:45:14.072578 14370 webserver.cc:460] Webserver started at http://127.30.208.62:34869/ using document root <none> and password file <none>
I20260807 19:45:14.073385 14370 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:14.073520 14370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:14.073850 14370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:14.077438 14370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "f9bc628fe0ca487b9b9b54b4a34c3343"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-vq2q"
server_key: "9b5e8bcfb16fb1c99e629dbb3b3808ba"
server_key_iv: "69729eb0354c04256e2242de9422fd14"
server_key_version: "encryptionkey@0"
I20260807 19:45:14.078295 14370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "f9bc628fe0ca487b9b9b54b4a34c3343"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-vq2q"
server_key: "9b5e8bcfb16fb1c99e629dbb3b3808ba"
server_key_iv: "69729eb0354c04256e2242de9422fd14"
server_key_version: "encryptionkey@0"
I20260807 19:45:14.084010 14370 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:45:14.087718 14387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:14.089111 14370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 19:45:14.089349 14370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "f9bc628fe0ca487b9b9b54b4a34c3343"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-vq2q"
server_key: "9b5e8bcfb16fb1c99e629dbb3b3808ba"
server_key_iv: "69729eb0354c04256e2242de9422fd14"
server_key_version: "encryptionkey@0"
I20260807 19:45:14.089550 14370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:14.112797 14370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:14.117372 14370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:14.117645 14370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:14.136050 14370 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:33657
I20260807 19:45:14.136108 14438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:33657 every 8 connection(s)
I20260807 19:45:14.137562 14370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:45:14.143150 14439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:14.145428 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 14370
I20260807 19:45:14.145689 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:45:14.146647 31552 external_mini_cluster.cc:1428] Setting key b174a1e59b459be3b448b79111122290
I20260807 19:45:14.156514 14439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Bootstrap starting.
I20260807 19:45:14.160477 14439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:14.161855 14439 log.cc:824] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:45:14 dist-test-slave-vq2q krb5kdc[14354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:14.165478 14439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: No bootstrap required, opened a new log
I20260807 19:45:14.172374 14442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:14.148976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53126 (local address 127.30.208.62:33657)
0807 19:45:14.149883 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:14.149895 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:14.149940 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:45:14.151595 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:14.151618 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:14.152008 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:14.152325 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:14.154132 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:14.155626 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:14.157460 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:14.158159 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:14.158341 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:14.163459 (+ 5118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:14.163471 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:14.163491 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:14.163543 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:14.167390 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:14.168175 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:14.168184 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:14.168193 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:14.168433 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:14.168774 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:14.168779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:14.168783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:14.169553 (+ 770us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:14.169992 (+ 439us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:14.170483 (+ 491us) server_negotiation.cc:299] Negotiation successful
0807 19:45:14.170962 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":417,"threads_started":1}
I20260807 19:45:14.175745 14439 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } }
I20260807 19:45:14.176100 14439 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:14.176174 14439 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9bc628fe0ca487b9b9b54b4a34c3343, State: Initialized, Role: FOLLOWER
I20260807 19:45:14.176656 14439 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } }
I20260807 19:45:14.176864 14439 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:45:14.176968 14439 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:45:14.177095 14439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:14.179574 14439 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } }
I20260807 19:45:14.180052 14439 leader_election.cc:302] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9bc628fe0ca487b9b9b54b4a34c3343; no voters:
I20260807 19:45:14.180609 14439 leader_election.cc:288] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:45:14.180747 14445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:45:14.181869 14445 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 1 LEADER]: Becoming Leader. State: Replica: f9bc628fe0ca487b9b9b54b4a34c3343, State: Running, Role: LEADER
I20260807 19:45:14.182538 14445 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } }
I20260807 19:45:14.183254 14439 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:45:14.185601 14448 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9bc628fe0ca487b9b9b54b4a34c3343. Latest consensus state: current_term: 1 leader_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } } }
I20260807 19:45:14.185570 14447 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } } }
I20260807 19:45:14.185993 14448 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:14.186025 14447 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:14.189383 14455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:45:14.194512 14455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:45:14.203225 14455 catalog_manager.cc:1347] Generated new cluster ID: ed858bbd30a94d318c6f9748023128ba
I20260807 19:45:14.203361 14455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:45:14.225512 14455 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:45:14.226913 14455 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:45:14.235036 14455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Generated new TSK 0
I20260807 19:45:14.236191 14455 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:45:14.313486 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33657
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:14.658740 14471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:14.659045 14471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:14.659168 14471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:14.666733 14471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:14.666859 14471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:14.667011 14471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:45:14.677663 14471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:14.679054 14471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:14.680403 14471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:14.693638 14478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:14.693815 14477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:14.693770 14481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:14.694571 14471 server_base.cc:1034] running on GCE node
I20260807 19:45:14.695168 14471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:14.696103 14471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:14.697290 14471 hybrid_clock.cc:648] HybridClock initialized: now 1786131914697240 us; error 38 us; skew 500 ppm
I20260807 19:45:14.697592 14471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:14 dist-test-slave-vq2q krb5kdc[14354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131914, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:14.700704 14471 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:45:14.702278 14471 webserver.cc:460] Webserver started at http://127.30.208.1:35067/ using document root <none> and password file <none>
I20260807 19:45:14.702973 14471 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:14.703096 14471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:14.703400 14471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:14.706485 14471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "3c13a30ed4d64ff788a1214b92d2b310"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-vq2q"
server_key: "598062ce90302438445bf5aaec791433"
server_key_iv: "aca159469318ee7f5a4723f881401598"
server_key_version: "encryptionkey@0"
I20260807 19:45:14.707310 14471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "3c13a30ed4d64ff788a1214b92d2b310"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-vq2q"
server_key: "598062ce90302438445bf5aaec791433"
server_key_iv: "aca159469318ee7f5a4723f881401598"
server_key_version: "encryptionkey@0"
I20260807 19:45:14.712538 14471 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:45:14.715931 14488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:14.717288 14471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:45:14.717440 14471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "3c13a30ed4d64ff788a1214b92d2b310"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-vq2q"
server_key: "598062ce90302438445bf5aaec791433"
server_key_iv: "aca159469318ee7f5a4723f881401598"
server_key_version: "encryptionkey@0"
I20260807 19:45:14.717630 14471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:14.735944 14471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:14.740129 14471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:14.740427 14471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:14.741504 14471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:14.742942 14471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:14.743021 14471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:14.743144 14471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:14.743197 14471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:14.775527 14471 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:40919
I20260807 19:45:14.775568 14608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:40919 every 8 connection(s)
I20260807 19:45:14.776777 14471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:45:14.783627 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 14471
I20260807 19:45:14.783818 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:45:14.784509 31552 external_mini_cluster.cc:1428] Setting key 73aa48e4ba1a0e126e71df80c6533e19
Aug 07 19:45:14 dist-test-slave-vq2q krb5kdc[14354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131914, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:14.797456 14611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:14.779227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:59911 (local address 127.30.208.62:33657)
0807 19:45:14.779673 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:14.779678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:14.780641 (+ 963us) server_negotiation.cc:407] Connection header received
0807 19:45:14.782558 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:14.782564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:14.782645 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:14.782743 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:14.784552 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:14.785536 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:14.787430 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:14.787865 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:14.787982 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:14.791787 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:14.791793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:14.791796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:14.791832 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:14.794390 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:14.795131 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:14.795137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:14.795140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:14.795264 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:14.795700 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:14.795704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:14.795706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:14.795991 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:14.796128 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:14.796948 (+ 820us) server_negotiation.cc:299] Negotiation successful
0807 19:45:14.797130 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":235,"threads_started":1}
I20260807 19:45:14.798910 14612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:14.779629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:59911)
0807 19:45:14.780454 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:14.780501 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:14.782285 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:14.782999 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:14.783007 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:14.783591 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:14.784374 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:14.784400 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:14.785731 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:14.785736 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:14.787266 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:14.787280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:14.788105 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:14.788113 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:14.788390 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:14.789091 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:14.789117 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:14.791532 (+ 2415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:14.794546 (+ 3014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:14.794557 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:14.794575 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:14.795009 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:14.795381 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:14.795386 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:14.795389 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:14.795588 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:14.796121 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:14.796130 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:14.796486 (+ 356us) client_negotiation.cc:769] Sending connection context
0807 19:45:14.796816 (+ 330us) client_negotiation.cc:240] Negotiation successful
0807 19:45:14.797420 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":372,"threads_started":1}
I20260807 19:45:14.800577 14609 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33657
I20260807 19:45:14.800913 14609 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:14.801872 14609 heartbeater.cc:511] Master 127.30.208.62:33657 requested a full tablet report, sending...
I20260807 19:45:14.804117 14404 ts_manager.cc:194] Registered new tserver with Master: 3c13a30ed4d64ff788a1214b92d2b310 (127.30.208.1:40919)
I20260807 19:45:14.805651 14404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:59911
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:45:14.848551 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33657
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:15.186475 14615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:15.186789 14615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:15.186931 14615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:15.194382 14615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:15.194515 14615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:15.194641 14615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:45:15.205304 14615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:15.206722 14615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:15.208061 14615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:15.221544 14623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:15.221589 14620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:15.221637 14621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:15.222164 14615 server_base.cc:1034] running on GCE node
I20260807 19:45:15.222815 14615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:15.223841 14615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:15.225076 14615 hybrid_clock.cc:648] HybridClock initialized: now 1786131915225028 us; error 38 us; skew 500 ppm
I20260807 19:45:15.225440 14615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:15 dist-test-slave-vq2q krb5kdc[14354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131915, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:15.228791 14615 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:45:15.230304 14615 webserver.cc:460] Webserver started at http://127.30.208.2:40531/ using document root <none> and password file <none>
I20260807 19:45:15.230997 14615 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:15.231103 14615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:15.231369 14615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:15.234403 14615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e"
format_stamp: "Formatted at 2026-08-07 19:45:15 on dist-test-slave-vq2q"
server_key: "7b79565c778fbcc07f52eb1c18cdec94"
server_key_iv: "c881a631fe427dae4ae41d4b69bf2dfe"
server_key_version: "encryptionkey@0"
I20260807 19:45:15.235252 14615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e"
format_stamp: "Formatted at 2026-08-07 19:45:15 on dist-test-slave-vq2q"
server_key: "7b79565c778fbcc07f52eb1c18cdec94"
server_key_iv: "c881a631fe427dae4ae41d4b69bf2dfe"
server_key_version: "encryptionkey@0"
I20260807 19:45:15.240875 14615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260807 19:45:15.244606 14633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:15.246104 14615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:45:15.246280 14615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e"
format_stamp: "Formatted at 2026-08-07 19:45:15 on dist-test-slave-vq2q"
server_key: "7b79565c778fbcc07f52eb1c18cdec94"
server_key_iv: "c881a631fe427dae4ae41d4b69bf2dfe"
server_key_version: "encryptionkey@0"
I20260807 19:45:15.246492 14615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:15.273550 14615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:15.277575 14615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:15.277863 14615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:15.278833 14615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:15.280210 14615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:15.280282 14615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:15.280385 14615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:15.280439 14615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:15.313467 14615 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:45655
I20260807 19:45:15.313520 14759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:45655 every 8 connection(s)
I20260807 19:45:15.314740 14615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:45:15.315905 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 14615
I20260807 19:45:15.316092 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:45:15.316670 31552 external_mini_cluster.cc:1428] Setting key 51537c765da596ea5578c13632e7c6be
Aug 07 19:45:15 dist-test-slave-vq2q krb5kdc[14354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131915, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:15.335927 14762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.317646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:47789 (local address 127.30.208.62:33657)
0807 19:45:15.318134 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:15.318139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:15.319247 (+ 1108us) server_negotiation.cc:407] Connection header received
0807 19:45:15.321145 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:15.321151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:15.321253 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:15.321356 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:15.322917 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.323868 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.325418 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.325992 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.326116 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:15.329930 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:15.329935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:15.329938 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:15.329975 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:15.332595 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:15.333547 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:15.333551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:15.333553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:15.333687 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:15.334138 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:15.334141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:15.334143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:15.334421 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:15.334529 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:15.335473 (+ 944us) server_negotiation.cc:299] Negotiation successful
0807 19:45:15.335653 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":276,"threads_started":1}
I20260807 19:45:15.337317 14763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.318210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:47789)
0807 19:45:15.319082 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:15.319127 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:15.320887 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:15.321589 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:15.321597 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:15.322099 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:15.322752 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:15.322770 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.324011 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.324014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:15.325276 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:15.325296 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.326237 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.326242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:15.326463 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:15.327155 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:15.327182 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:15.329656 (+ 2474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:15.332762 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:15.332771 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:15.332788 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:15.333443 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:15.333787 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:15.333791 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:15.333794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:15.334039 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:15.334543 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:15.334551 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:15.335014 (+ 463us) client_negotiation.cc:769] Sending connection context
0807 19:45:15.335356 (+ 342us) client_negotiation.cc:240] Negotiation successful
0807 19:45:15.335907 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":329,"threads_started":1}
I20260807 19:45:15.338940 14760 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33657
I20260807 19:45:15.339257 14760 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:15.340047 14760 heartbeater.cc:511] Master 127.30.208.62:33657 requested a full tablet report, sending...
I20260807 19:45:15.341630 14404 ts_manager.cc:194] Registered new tserver with Master: bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655)
I20260807 19:45:15.342828 14404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:47789
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:45:15.378783 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33657
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:15.716595 14766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:15.716892 14766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:15.717016 14766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:15.725112 14766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:15.725299 14766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:15.725482 14766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:45:15.735421 14766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:15.737048 14766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:15.738773 14766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:15.751652 14771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:15.751742 14772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:15.752229 14774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:15.752657 14766 server_base.cc:1034] running on GCE node
I20260807 19:45:15.753331 14766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:15.754354 14766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:15.755584 14766 hybrid_clock.cc:648] HybridClock initialized: now 1786131915755516 us; error 52 us; skew 500 ppm
I20260807 19:45:15.755935 14766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:15 dist-test-slave-vq2q krb5kdc[14354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131915, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:15.759107 14766 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:45:15.760615 14766 webserver.cc:460] Webserver started at http://127.30.208.3:44205/ using document root <none> and password file <none>
I20260807 19:45:15.761389 14766 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:15.761515 14766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:15.761807 14766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:15.764750 14766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "43ea70e9351043a19cec0a410e071558"
format_stamp: "Formatted at 2026-08-07 19:45:15 on dist-test-slave-vq2q"
server_key: "e63f83b44c200eab4e13dce916cbb86b"
server_key_iv: "fe149cf6462f00f6a1ef29c1a31a55ce"
server_key_version: "encryptionkey@0"
I20260807 19:45:15.765601 14766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "43ea70e9351043a19cec0a410e071558"
format_stamp: "Formatted at 2026-08-07 19:45:15 on dist-test-slave-vq2q"
server_key: "e63f83b44c200eab4e13dce916cbb86b"
server_key_iv: "fe149cf6462f00f6a1ef29c1a31a55ce"
server_key_version: "encryptionkey@0"
I20260807 19:45:15.770915 14766 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:45:15.774609 14782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:15.776527 14766 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 19:45:15.776691 14766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "43ea70e9351043a19cec0a410e071558"
format_stamp: "Formatted at 2026-08-07 19:45:15 on dist-test-slave-vq2q"
server_key: "e63f83b44c200eab4e13dce916cbb86b"
server_key_iv: "fe149cf6462f00f6a1ef29c1a31a55ce"
server_key_version: "encryptionkey@0"
I20260807 19:45:15.776897 14766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:15.790700 14766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:15.795100 14766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:15.795380 14766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:15.796329 14766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:15.797703 14766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:15.797782 14766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:15.797899 14766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:15.797948 14766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:15.809262 14609 heartbeater.cc:503] Master 127.30.208.62:33657 was elected leader, sending a full tablet report...
I20260807 19:45:15.828299 14766 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:40887
I20260807 19:45:15.828349 14906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:40887 every 8 connection(s)
I20260807 19:45:15.829607 14766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:45:15.835985 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 14766
I20260807 19:45:15.836171 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:45:15.836721 31552 external_mini_cluster.cc:1428] Setting key cc15a99e660a24816439f6c33ce19241
Aug 07 19:45:15 dist-test-slave-vq2q krb5kdc[14354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131915, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:15.851056 14762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.832589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:48559 (local address 127.30.208.62:33657)
0807 19:45:15.832811 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:15.832818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:15.834277 (+ 1459us) server_negotiation.cc:407] Connection header received
0807 19:45:15.836201 (+ 1924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:15.836208 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:15.836306 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:15.836446 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:15.838276 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.839166 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.840944 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.841417 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.841522 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:15.845123 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:15.845128 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:15.845132 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:15.845167 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:15.847791 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:15.848628 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:15.848633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:15.848635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:15.848781 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:15.849249 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:15.849252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:15.849255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:15.849585 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:15.849726 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:15.850517 (+ 791us) server_negotiation.cc:299] Negotiation successful
0807 19:45:15.850785 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:45:15.852361 14910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.833286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:48559)
0807 19:45:15.834099 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:15.834150 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:15.835883 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:15.836670 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:15.836679 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:15.837326 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:15.838089 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:15.838119 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.839326 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.839332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:15.840802 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:15.840821 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.841658 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.841665 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:15.841887 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:15.842536 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:15.842563 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:15.844897 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:15.847979 (+ 3082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:15.847986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:15.848000 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:15.848486 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:15.848889 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:15.848892 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:15.848894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:15.849090 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:15.849707 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:15.849713 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:15.850067 (+ 354us) client_negotiation.cc:769] Sending connection context
0807 19:45:15.850406 (+ 339us) client_negotiation.cc:240] Negotiation successful
0807 19:45:15.850915 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":315,"threads_started":1}
I20260807 19:45:15.853731 14907 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33657
I20260807 19:45:15.854072 14907 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:15.854782 14907 heartbeater.cc:511] Master 127.30.208.62:33657 requested a full tablet report, sending...
I20260807 19:45:15.856244 14404 ts_manager.cc:194] Registered new tserver with Master: 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887)
I20260807 19:45:15.857074 14404 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:48559
I20260807 19:45:15.865863 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:45:15.883657 14762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.870419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53132 (local address 127.30.208.62:33657)
0807 19:45:15.870627 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:15.870632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:15.871115 (+ 483us) server_negotiation.cc:407] Connection header received
0807 19:45:15.871226 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:15.871230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:15.871304 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:15.871396 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:15.872870 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.873800 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.875171 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.875592 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.875691 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:15.878149 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:15.878155 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:15.878159 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:15.878196 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:15.880849 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:15.881590 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:15.881595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:15.881598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:15.881744 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:15.882173 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:15.882177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:15.882179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:15.882556 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:15.882766 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:15.883160 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 19:45:15.883364 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:45:15.888253 14404 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:45:15.896030 14404 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:45:15.919822 14921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.908447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:40919 (local address 127.0.0.1:53818)
0807 19:45:15.909095 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:15.909114 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:15.909301 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:15.910617 (+ 1316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:15.910627 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:15.910663 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:15.911119 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:15.911137 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.913581 (+ 2444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.913586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:15.916186 (+ 2600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:15.916209 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.918481 (+ 2272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.918498 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:15.918817 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:15.918896 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 19:45:15.919049 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 19:45:15.919275 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"spinlock_wait_cycles":17664,"thread_start_us":766,"threads_started":1}
I20260807 19:45:15.921716 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.909009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53818 (local address 127.30.208.1:40919)
0807 19:45:15.909905 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:15.909914 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:15.909944 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:45:15.910044 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:15.910054 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:15.910342 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:15.911357 (+ 1015us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:15.911669 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.913421 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.916450 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.918091 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.920799 (+ 2708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:15.921095 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:15.921199 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 19:45:15.921382 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":658,"threads_started":1}
I20260807 19:45:15.927359 14541 tablet_service.cc:1467] Processing CreateTablet for tablet f710a807a80f4877b47a426c6d4696f2 (DEFAULT_TABLE table=test-table [id=0fcff52e166b4a76a5ce50db631459a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:15.929014 14541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f710a807a80f4877b47a426c6d4696f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:15.930397 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.908489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50154 (local address 127.30.208.3:40887)
0807 19:45:15.923256 (+ 14767us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:15.923266 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:15.923302 (+ 36us) server_negotiation.cc:407] Connection header received
0807 19:45:15.923402 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:15.923410 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:15.923681 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:15.923898 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:15.924675 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.925844 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.928050 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.929298 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.929429 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:15.929664 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:15.929894 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 19:45:15.930069 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14543,"thread_start_us":14390,"threads_started":1}
I20260807 19:45:15.930423 14920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.908125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:40887 (local address 127.0.0.1:50154)
0807 19:45:15.909038 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:15.909086 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:15.909301 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:15.924061 (+ 14760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:15.924066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:15.924096 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:15.924480 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:15.924489 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.925992 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.925997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:15.927887 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:15.927900 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.929511 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.929518 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:15.929692 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:15.929726 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:45:15.929829 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 19:45:15.929977 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"spinlock_wait_cycles":3840,"thread_start_us":544,"threads_started":1}
I20260807 19:45:15.931077 14921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.909556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:45655 (local address 127.0.0.1:43740)
0807 19:45:15.920381 (+ 10825us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:15.920406 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:15.920530 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:15.924183 (+ 3653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:15.924188 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:15.924234 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:15.924526 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:15.924539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.926642 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.926651 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:15.928516 (+ 1865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:15.928533 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.930267 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.930273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:15.930452 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:15.930485 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:45:15.930552 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 19:45:15.930669 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10703,"spinlock_wait_cycles":1792,"thread_start_us":12965,"threads_started":1}
I20260807 19:45:15.931413 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.909801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43740 (local address 127.30.208.2:45655)
0807 19:45:15.923600 (+ 13799us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:15.923607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:15.923631 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:45:15.923709 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:15.923715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:15.923909 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:15.924723 (+ 814us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:15.924962 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.926445 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.928694 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.930069 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.930768 (+ 699us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:15.930960 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:15.931026 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 19:45:15.931182 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13639,"thread_start_us":13500,"threads_started":1}
I20260807 19:45:15.934628 14690 tablet_service.cc:1467] Processing CreateTablet for tablet f710a807a80f4877b47a426c6d4696f2 (DEFAULT_TABLE table=test-table [id=0fcff52e166b4a76a5ce50db631459a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:15.934722 14839 tablet_service.cc:1467] Processing CreateTablet for tablet f710a807a80f4877b47a426c6d4696f2 (DEFAULT_TABLE table=test-table [id=0fcff52e166b4a76a5ce50db631459a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:15.935813 14690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f710a807a80f4877b47a426c6d4696f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:15.936096 14839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f710a807a80f4877b47a426c6d4696f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:15.944083 14927 tablet_bootstrap.cc:492] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310: Bootstrap starting.
I20260807 19:45:15.947889 14927 tablet_bootstrap.cc:654] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:15.948348 14928 tablet_bootstrap.cc:492] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e: Bootstrap starting.
I20260807 19:45:15.950111 14927 log.cc:824] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:15.950730 14929 tablet_bootstrap.cc:492] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558: Bootstrap starting.
I20260807 19:45:15.952085 14928 tablet_bootstrap.cc:654] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:15.953123 14928 log.cc:824] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:15.954280 14929 tablet_bootstrap.cc:654] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:15.954346 14927 tablet_bootstrap.cc:492] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310: No bootstrap required, opened a new log
I20260807 19:45:15.954627 14927 ts_tablet_manager.cc:1397] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260807 19:45:15.955551 14929 log.cc:824] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:15.955705 14928 tablet_bootstrap.cc:492] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e: No bootstrap required, opened a new log
I20260807 19:45:15.955953 14928 ts_tablet_manager.cc:1397] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:45:15.958746 14929 tablet_bootstrap.cc:492] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558: No bootstrap required, opened a new log
I20260807 19:45:15.959021 14929 ts_tablet_manager.cc:1397] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:45:15.963724 14928 raft_consensus.cc:348] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } } peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } }
I20260807 19:45:15.964128 14928 raft_consensus.cc:374] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:15.964221 14928 raft_consensus.cc:729] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf33209f9b2e4d76a3ca7fecebeeed1e, State: Initialized, Role: FOLLOWER
I20260807 19:45:15.964807 14928 consensus_queue.cc:260] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } } peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } }
I20260807 19:45:15.964666 14927 raft_consensus.cc:348] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } } peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } }
I20260807 19:45:15.965077 14927 raft_consensus.cc:374] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:15.965169 14927 raft_consensus.cc:729] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c13a30ed4d64ff788a1214b92d2b310, State: Initialized, Role: FOLLOWER
I20260807 19:45:15.965992 14760 heartbeater.cc:503] Master 127.30.208.62:33657 was elected leader, sending a full tablet report...
I20260807 19:45:15.965905 14927 consensus_queue.cc:260] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } } peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } }
I20260807 19:45:15.967777 14927 ts_tablet_manager.cc:1428] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310: Time spent starting tablet: real 0.013s user 0.011s sys 0.003s
I20260807 19:45:15.969102 14929 raft_consensus.cc:348] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } } peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } }
I20260807 19:45:15.969501 14929 raft_consensus.cc:374] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:15.969599 14929 raft_consensus.cc:729] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43ea70e9351043a19cec0a410e071558, State: Initialized, Role: FOLLOWER
I20260807 19:45:15.970193 14928 ts_tablet_manager.cc:1428] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260807 19:45:15.970175 14929 consensus_queue.cc:260] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } } peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } }
I20260807 19:45:15.971302 14907 heartbeater.cc:503] Master 127.30.208.62:33657 was elected leader, sending a full tablet report...
I20260807 19:45:15.973217 14929 ts_tablet_manager.cc:1428] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260807 19:45:15.993149 14934 raft_consensus.cc:482] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:15.993575 14934 raft_consensus.cc:504] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } } peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } }
I20260807 19:45:15.995321 14934 leader_election.cc:288] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887), bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655)
I20260807 19:45:16.004650 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.995760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:47457 (local address 127.30.208.3:40887)
0807 19:45:15.995983 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:15.995989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:15.996701 (+ 712us) server_negotiation.cc:407] Connection header received
0807 19:45:15.996951 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:15.996956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:15.997029 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:15.997107 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:15.998000 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.999091 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:16.002174 (+ 3083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:16.003440 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:16.003613 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:16.003689 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:16.004260 (+ 571us) server_negotiation.cc:299] Negotiation successful
0807 19:45:16.004388 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 19:45:16.004650 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.995910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:44219 (local address 127.30.208.2:45655)
0807 19:45:15.996133 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:15.996140 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:15.996699 (+ 559us) server_negotiation.cc:407] Connection header received
0807 19:45:15.996888 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:15.996894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:15.996973 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:15.997069 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:15.998000 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.999102 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:16.002164 (+ 3062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:16.003466 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:16.003612 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:16.003700 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:16.004253 (+ 553us) server_negotiation.cc:299] Negotiation successful
0807 19:45:16.004388 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 19:45:16.004884 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.995550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:40887 (local address 127.30.208.1:47457)
0807 19:45:15.996566 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:15.996590 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:15.996809 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:15.997151 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:15.997155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:15.997243 (+ 88us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:15.997842 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:15.997851 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.999248 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.999253 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:16.001943 (+ 2690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:16.001965 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:16.003757 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:16.003765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:16.003994 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:16.004107 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 19:45:16.004228 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 19:45:16.004370 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":280,"spinlock_wait_cycles":67968,"thread_start_us":603,"threads_started":1}
I20260807 19:45:16.004906 14938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:15.995550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:45655 (local address 127.30.208.1:44219)
0807 19:45:15.996601 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:15.996622 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:15.996751 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:15.997144 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:15.997149 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:15.997244 (+ 95us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:15.997817 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:15.997836 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:15.999247 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:15.999253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:16.001943 (+ 2690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:16.001965 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:16.003757 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:16.003765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:16.003996 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:16.004105 (+ 109us) client_negotiation.cc:769] Sending connection context
0807 19:45:16.004203 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 19:45:16.004377 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"mutex_wait_us":254,"spinlock_wait_cycles":19840,"thread_start_us":525,"threads_started":1}
I20260807 19:45:16.006086 14860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f710a807a80f4877b47a426c6d4696f2" candidate_uuid: "3c13a30ed4d64ff788a1214b92d2b310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43ea70e9351043a19cec0a410e071558" is_pre_election: true
I20260807 19:45:16.006067 14710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f710a807a80f4877b47a426c6d4696f2" candidate_uuid: "3c13a30ed4d64ff788a1214b92d2b310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" is_pre_election: true
I20260807 19:45:16.006675 14710 raft_consensus.cc:2454] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c13a30ed4d64ff788a1214b92d2b310 in term 0.
I20260807 19:45:16.006681 14860 raft_consensus.cc:2454] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c13a30ed4d64ff788a1214b92d2b310 in term 0.
I20260807 19:45:16.007581 14489 leader_election.cc:302] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3c13a30ed4d64ff788a1214b92d2b310, 43ea70e9351043a19cec0a410e071558, bf33209f9b2e4d76a3ca7fecebeeed1e; no voters:
I20260807 19:45:16.008065 14934 raft_consensus.cc:2781] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:45:16.008200 14934 raft_consensus.cc:482] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:45:16.008307 14934 raft_consensus.cc:3037] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:16.010944 14934 raft_consensus.cc:504] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } } peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } }
I20260807 19:45:16.011564 14934 leader_election.cc:288] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 1 election: Requested vote from peers 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887), bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655)
I20260807 19:45:16.012053 14860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f710a807a80f4877b47a426c6d4696f2" candidate_uuid: "3c13a30ed4d64ff788a1214b92d2b310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43ea70e9351043a19cec0a410e071558"
I20260807 19:45:16.012065 14710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f710a807a80f4877b47a426c6d4696f2" candidate_uuid: "3c13a30ed4d64ff788a1214b92d2b310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e"
I20260807 19:45:16.012241 14860 raft_consensus.cc:3037] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:16.012256 14710 raft_consensus.cc:3037] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:16.014557 14860 raft_consensus.cc:2454] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c13a30ed4d64ff788a1214b92d2b310 in term 1.
I20260807 19:45:16.014564 14710 raft_consensus.cc:2454] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c13a30ed4d64ff788a1214b92d2b310 in term 1.
I20260807 19:45:16.014989 14489 leader_election.cc:302] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c13a30ed4d64ff788a1214b92d2b310, bf33209f9b2e4d76a3ca7fecebeeed1e; no voters:
I20260807 19:45:16.015298 14934 raft_consensus.cc:2781] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:45:16.015755 14934 raft_consensus.cc:686] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 LEADER]: Becoming Leader. State: Replica: 3c13a30ed4d64ff788a1214b92d2b310, State: Running, Role: LEADER
I20260807 19:45:16.016300 14934 consensus_queue.cc:237] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } } peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } }
I20260807 19:45:16.021247 14403 catalog_manager.cc:5495] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c13a30ed4d64ff788a1214b92d2b310 (127.30.208.1). New cstate: current_term: 1 leader_uuid: "3c13a30ed4d64ff788a1214b92d2b310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } health_report { overall_health: UNKNOWN } } }
W20260807 19:45:16.029083 14610 tablet.cc:2369] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:45:16.046586 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:16.041125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53832 (local address 127.30.208.1:40919)
0807 19:45:16.041345 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:16.041350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:16.041368 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:45:16.041454 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:16.041458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:16.041536 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:16.041670 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:45:16.041989 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:16.042698 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:16.044191 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:16.044745 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:16.044856 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:16.045429 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:45:16.045992 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:45:16.046107 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:16.046250 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 19:45:16.046324 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:45:16.054883 14710 raft_consensus.cc:1261] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e [term 1 FOLLOWER]: Refusing update from remote peer 3c13a30ed4d64ff788a1214b92d2b310: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:45:16.054922 14860 raft_consensus.cc:1261] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558 [term 1 FOLLOWER]: Refusing update from remote peer 3c13a30ed4d64ff788a1214b92d2b310: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:45:16.055871 14939 consensus_queue.cc:1035] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:45:16.056186 14934 consensus_queue.cc:1035] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:45:16.067540 14761 tablet.cc:2369] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:45:16.067981 14943 mvcc.cc:205] Tried to move back new op lower bound from 7315996328151011328 to 7315996328007045120. Current Snapshot: MvccSnapshot[applied={T|T < 7315996328151011328}]
I20260807 19:45:16.068840 14947 mvcc.cc:205] Tried to move back new op lower bound from 7315996328151011328 to 7315996328007045120. Current Snapshot: MvccSnapshot[applied={T|T < 7315996328151011328}]
I20260807 19:45:16.082326 14403 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:45:16.082645 14403 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:45:16.086481 14403 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Sending DeleteTablet for 3 replicas of tablet f710a807a80f4877b47a426c6d4696f2
I20260807 19:45:16.087769 14839 tablet_service.cc:1514] Processing DeleteTablet for tablet f710a807a80f4877b47a426c6d4696f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:16 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:50154
I20260807 19:45:16.087896 14541 tablet_service.cc:1514] Processing DeleteTablet for tablet f710a807a80f4877b47a426c6d4696f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:16 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:53818
I20260807 19:45:16.088181 14690 tablet_service.cc:1514] Processing DeleteTablet for tablet f710a807a80f4877b47a426c6d4696f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:16 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:43740
I20260807 19:45:16.089332 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 14471
I20260807 19:45:16.097915 14955 tablet_replica.cc:330] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558: stopping tablet replica
I20260807 19:45:16.098492 14955 raft_consensus.cc:2229] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:45:16.099125 14955 raft_consensus.cc:2258] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:16.100001 14957 tablet_replica.cc:330] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e: stopping tablet replica
I20260807 19:45:16.100517 14957 raft_consensus.cc:2229] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:45:16.101095 14957 raft_consensus.cc:2258] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:16.103541 14955 ts_tablet_manager.cc:1905] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:45:16.104650 14956 tablet_replica.cc:330] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310: stopping tablet replica
I20260807 19:45:16.105127 14956 raft_consensus.cc:2229] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:45:16.105930 14956 raft_consensus.cc:2258] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:16.108439 14956 ts_tablet_manager.cc:1905] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:45:16.110459 14955 ts_tablet_manager.cc:1918] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:45:16.110657 14955 log.cc:1196] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/f710a807a80f4877b47a426c6d4696f2
I20260807 19:45:16.111248 14955 ts_tablet_manager.cc:1939] T f710a807a80f4877b47a426c6d4696f2 P 43ea70e9351043a19cec0a410e071558: Deleting consensus metadata
I20260807 19:45:16.113354 14391 catalog_manager.cc:4841] TS 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887): tablet f710a807a80f4877b47a426c6d4696f2 (table test-table [id=0fcff52e166b4a76a5ce50db631459a3]) successfully deleted
I20260807 19:45:16.115213 14957 ts_tablet_manager.cc:1905] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:45:16.115875 14956 ts_tablet_manager.cc:1918] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:45:16.116055 14956 log.cc:1196] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/f710a807a80f4877b47a426c6d4696f2
Aug 07 19:45:16 dist-test-slave-vq2q krb5kdc[14354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:45:16.116637 14956 ts_tablet_manager.cc:1939] T f710a807a80f4877b47a426c6d4696f2 P 3c13a30ed4d64ff788a1214b92d2b310: Deleting consensus metadata
I20260807 19:45:16.118629 14388 catalog_manager.cc:4841] TS 3c13a30ed4d64ff788a1214b92d2b310 (127.30.208.1:40919): tablet f710a807a80f4877b47a426c6d4696f2 (table test-table [id=0fcff52e166b4a76a5ce50db631459a3]) successfully deleted
I20260807 19:45:16.122408 14957 ts_tablet_manager.cc:1918] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:45:16.122586 14957 log.cc:1196] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/f710a807a80f4877b47a426c6d4696f2
I20260807 19:45:16.123157 14957 ts_tablet_manager.cc:1939] T f710a807a80f4877b47a426c6d4696f2 P bf33209f9b2e4d76a3ca7fecebeeed1e: Deleting consensus metadata
I20260807 19:45:16.124104 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:16.090643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53844 (local address 127.30.208.1:40919)
0807 19:45:16.090832 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:16.090839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:16.105115 (+ 14276us) server_negotiation.cc:407] Connection header received
0807 19:45:16.105575 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:16.105581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:16.105709 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:16.105800 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:16.108037 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:16.109207 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:16.111182 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:16.111955 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:16.112065 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:16.117567 (+ 5502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:16.117581 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:16.117600 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:16.117650 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:16.121033 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:16.121843 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:16.121854 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:16.121857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:16.122034 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:16.122420 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:16.122425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:16.122427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:16.122867 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:16.123155 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:16.123514 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 19:45:16.123749 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:45:16.124752 14604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:16.125063 14388 catalog_manager.cc:4841] TS bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655): tablet f710a807a80f4877b47a426c6d4696f2 (table test-table [id=0fcff52e166b4a76a5ce50db631459a3]) successfully deleted
I20260807 19:45:16.224280 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 14471
I20260807 19:45:16.242628 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 14615
Aug 07 19:45:16 dist-test-slave-vq2q krb5kdc[14354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:45:16.257666 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:16.243411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43750 (local address 127.30.208.2:45655)
0807 19:45:16.243589 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:16.243594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:16.243614 (+ 20us) server_negotiation.cc:407] Connection header received
0807 19:45:16.243791 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:16.243797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:16.243869 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:16.243966 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:16.245278 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:16.246035 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:16.247471 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:16.247862 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:16.247957 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:16.252413 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:16.252429 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:16.252444 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:16.252480 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:16.255178 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:16.255852 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:16.255859 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:16.255862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:16.255983 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:16.256373 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:16.256376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:16.256379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:16.256733 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:16.256958 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:16.257214 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 19:45:16.257411 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:45:16.258373 14754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:16.348297 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 14615
I20260807 19:45:16.366264 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 14766
Aug 07 19:45:16 dist-test-slave-vq2q krb5kdc[14354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:45:16.382280 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:16.367063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50162 (local address 127.30.208.3:40887)
0807 19:45:16.367255 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:16.367261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:16.367283 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:45:16.367427 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:16.367432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:16.367516 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:16.367614 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:16.369021 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:16.369812 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:16.371257 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:16.371655 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:16.371782 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:16.376352 (+ 4570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:16.376369 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:16.376385 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:16.376423 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:16.379260 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:16.380019 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:16.380028 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:16.380030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:16.380195 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:16.380622 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:16.380626 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:16.380628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:16.381025 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:16.381346 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:16.381700 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 19:45:16.381954 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:45:16.382974 14902 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:16.472851 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 14766
I20260807 19:45:16.490787 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 14370
I20260807 19:45:16.491957 14434 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:16.604139 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 14370
I20260807 19:45:16.626456 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33657
--webserver_interface=127.30.208.62
--webserver_port=34869
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:33657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:16.970016 14963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:16.970332 14963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:16.970468 14963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:16.977901 14963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:45:16.978034 14963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:16.978117 14963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:16.978194 14963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:45:16.978255 14963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:45:16.988438 14963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:33657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=34869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:16.989850 14963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:16.991190 14963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:17.015475 14969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:17.015502 14971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:17.015543 14973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:17.016711 14963 server_base.cc:1034] running on GCE node
I20260807 19:45:17.017721 14963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:17.019423 14963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:17.020654 14963 hybrid_clock.cc:648] HybridClock initialized: now 1786131917020600 us; error 45 us; skew 500 ppm
I20260807 19:45:17.021095 14963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:17 dist-test-slave-vq2q krb5kdc[14354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131917, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:17.024799 14963 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:45:17.026590 14963 webserver.cc:460] Webserver started at http://127.30.208.62:34869/ using document root <none> and password file <none>
I20260807 19:45:17.027402 14963 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:17.027547 14963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:17.033991 14963 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 19:45:17.037027 14981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:17.038419 14963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:45:17.038586 14963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "f9bc628fe0ca487b9b9b54b4a34c3343"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-vq2q"
server_key: "9b5e8bcfb16fb1c99e629dbb3b3808ba"
server_key_iv: "69729eb0354c04256e2242de9422fd14"
server_key_version: "encryptionkey@0"
I20260807 19:45:17.039706 14963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:17.075839 14963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:17.080255 14963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:17.080540 14963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:17.098977 14963 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:33657
I20260807 19:45:17.099015 15035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:33657 every 8 connection(s)
I20260807 19:45:17.100548 14963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:45:17.103578 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 14963
I20260807 19:45:17.104382 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:40919
--local_ip_for_outbound_sockets=127.30.208.1
--tserver_master_addrs=127.30.208.62:33657
--webserver_port=35067
--webserver_interface=127.30.208.1
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:45:17.108035 15036 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:45:17.111515 15036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Bootstrap starting.
I20260807 19:45:17.131812 15036 log.cc:824] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:17.144505 15036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:45:17.145031 15036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Bootstrap complete.
I20260807 19:45:17.154649 15036 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } }
I20260807 19:45:17.155647 15036 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9bc628fe0ca487b9b9b54b4a34c3343, State: Initialized, Role: FOLLOWER
I20260807 19:45:17.156279 15036 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } }
I20260807 19:45:17.156535 15036 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:45:17.156656 15036 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:45:17.156824 15036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:45:17.160935 15036 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } }
I20260807 19:45:17.161542 15036 leader_election.cc:302] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9bc628fe0ca487b9b9b54b4a34c3343; no voters:
I20260807 19:45:17.162158 15036 leader_election.cc:288] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:45:17.162307 15040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:45:17.164594 15036 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:45:17.169387 15040 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 2 LEADER]: Becoming Leader. State: Replica: f9bc628fe0ca487b9b9b54b4a34c3343, State: Running, Role: LEADER
I20260807 19:45:17.170028 15040 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } }
I20260807 19:45:17.172993 15042 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9bc628fe0ca487b9b9b54b4a34c3343. Latest consensus state: current_term: 2 leader_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } } }
I20260807 19:45:17.173411 15042 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:17.173359 15041 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } } }
I20260807 19:45:17.173682 15041 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:17.186585 15050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:45:17.195472 15050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:45:17.196982 15050 catalog_manager.cc:1259] Loaded cluster ID: ed858bbd30a94d318c6f9748023128ba
I20260807 19:45:17.197100 15050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:45:17.200431 15050 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 19:45:17.216820 15050 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 19:45:17.226565 15050 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:45:17.228464 15050 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Loaded TSK: 0
I20260807 19:45:17.230119 15050 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:45:17.468273 15038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:17.468606 15038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:17.468750 15038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:17.476372 15038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:17.476516 15038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:17.476672 15038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:45:17.487224 15038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:40919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=35067
--tserver_master_addrs=127.30.208.62:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:17.488539 15038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:17.490036 15038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:17.505095 15064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:17.505846 15065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:17.506103 15067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:17.506911 15038 server_base.cc:1034] running on GCE node
I20260807 19:45:17.507571 15038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:17.508589 15038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:17.509814 15038 hybrid_clock.cc:648] HybridClock initialized: now 1786131917509725 us; error 61 us; skew 500 ppm
I20260807 19:45:17.510123 15038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:17 dist-test-slave-vq2q krb5kdc[14354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131917, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:17.513417 15038 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:45:17.514977 15038 webserver.cc:460] Webserver started at http://127.30.208.1:35067/ using document root <none> and password file <none>
I20260807 19:45:17.515669 15038 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:17.515812 15038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:17.521503 15038 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:45:17.524518 15077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:17.526100 15038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:45:17.526265 15038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "3c13a30ed4d64ff788a1214b92d2b310"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-vq2q"
server_key: "598062ce90302438445bf5aaec791433"
server_key_iv: "aca159469318ee7f5a4723f881401598"
server_key_version: "encryptionkey@0"
I20260807 19:45:17.527302 15038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:17.547465 15038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:17.552273 15038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:17.552543 15038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:17.553581 15038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:17.554903 15038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:17.555008 15038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:17.555124 15038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:17.555195 15038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:17.586352 15038 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:40919
I20260807 19:45:17.586418 15207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:40919 every 8 connection(s)
I20260807 19:45:17.587795 15038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:45:17.593232 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 15038
I20260807 19:45:17.593886 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:45655
--local_ip_for_outbound_sockets=127.30.208.2
--tserver_master_addrs=127.30.208.62:33657
--webserver_port=40531
--webserver_interface=127.30.208.2
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:45:17 dist-test-slave-vq2q krb5kdc[14354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131917, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:17.610986 15212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:17.590781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:53609)
0807 19:45:17.591560 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:17.591605 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:17.593315 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:17.594753 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:17.594765 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:17.595366 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:17.595893 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:17.595911 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:17.597573 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:17.597577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:17.598769 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:17.598782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:17.599775 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:17.599782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:17.600049 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:17.600684 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:17.600706 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:17.602870 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:17.606751 (+ 3881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:17.606758 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:17.606769 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:17.607225 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:17.607727 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:17.607731 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:17.607733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:17.607946 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:17.608801 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:17.608808 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:17.609123 (+ 315us) client_negotiation.cc:769] Sending connection context
0807 19:45:17.609421 (+ 298us) client_negotiation.cc:240] Negotiation successful
0807 19:45:17.609818 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":314,"threads_started":1}
I20260807 19:45:17.611243 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:17.590569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:53609 (local address 127.30.208.62:33657)
0807 19:45:17.591216 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:17.591233 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:17.591741 (+ 508us) server_negotiation.cc:407] Connection header received
0807 19:45:17.593852 (+ 2111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:17.593865 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:17.594236 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:17.594577 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:17.596115 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:17.597402 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:17.598915 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:17.599540 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:17.599722 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:17.603140 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:17.603148 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:17.603163 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:17.603199 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:17.606584 (+ 3385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:17.607372 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:17.607381 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:17.607387 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:17.607591 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:17.608062 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:17.608066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:17.608069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:17.608661 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:17.608864 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:17.609661 (+ 797us) server_negotiation.cc:299] Negotiation successful
0807 19:45:17.610070 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":257,"threads_started":1}
I20260807 19:45:17.614161 15209 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33657
I20260807 19:45:17.614476 15209 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:17.615379 15209 heartbeater.cc:511] Master 127.30.208.62:33657 requested a full tablet report, sending...
I20260807 19:45:17.618062 15000 ts_manager.cc:194] Registered new tserver with Master: 3c13a30ed4d64ff788a1214b92d2b310 (127.30.208.1:40919)
I20260807 19:45:17.620223 15000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:53609
W20260807 19:45:17.958016 15213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:17.958310 15213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:17.958406 15213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:17.965713 15213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:17.965821 15213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:17.965946 15213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:45:17.975780 15213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:45655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=40531
--tserver_master_addrs=127.30.208.62:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:17.977020 15213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:17.978299 15213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:17.991420 15220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:17.991508 15223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:17.991559 15221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:17.993479 15213 server_base.cc:1034] running on GCE node
I20260807 19:45:17.994176 15213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:17.995234 15213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:17.996440 15213 hybrid_clock.cc:648] HybridClock initialized: now 1786131917996389 us; error 39 us; skew 500 ppm
I20260807 19:45:17.996762 15213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:17 dist-test-slave-vq2q krb5kdc[14354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131917, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:17.999951 15213 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:45:18.001477 15213 webserver.cc:460] Webserver started at http://127.30.208.2:40531/ using document root <none> and password file <none>
I20260807 19:45:18.002173 15213 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:18.002308 15213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:18.007819 15213 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:45:18.010542 15230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:18.012003 15213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:45:18.012172 15213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e"
format_stamp: "Formatted at 2026-08-07 19:45:15 on dist-test-slave-vq2q"
server_key: "7b79565c778fbcc07f52eb1c18cdec94"
server_key_iv: "c881a631fe427dae4ae41d4b69bf2dfe"
server_key_version: "encryptionkey@0"
I20260807 19:45:18.013234 15213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:18.040911 15213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:18.045517 15213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:18.045814 15213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:18.046846 15213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:18.048276 15213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:18.048393 15213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:18.048498 15213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:18.048569 15213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:18.079785 15213 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:45655
I20260807 19:45:18.079838 15358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:45655 every 8 connection(s)
I20260807 19:45:18.081292 15213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:45:18.084619 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 15213
I20260807 19:45:18.085315 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:40887
--local_ip_for_outbound_sockets=127.30.208.3
--tserver_master_addrs=127.30.208.62:33657
--webserver_port=44205
--webserver_interface=127.30.208.3
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:45:18 dist-test-slave-vq2q krb5kdc[14354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131917, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:18.102846 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.083861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:46121 (local address 127.30.208.62:33657)
0807 19:45:18.084063 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:18.084070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:18.085624 (+ 1554us) server_negotiation.cc:407] Connection header received
0807 19:45:18.087753 (+ 2129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:18.087759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:18.087846 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:18.087935 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:18.089522 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.090241 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.091868 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.092458 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.092609 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.096247 (+ 3638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:18.096252 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:18.096255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:18.096292 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:18.099219 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:18.100122 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:18.100127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:18.100131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:18.100292 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:18.100795 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:18.100800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:18.100804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:18.101209 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:18.101345 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:18.102238 (+ 893us) server_negotiation.cc:299] Negotiation successful
0807 19:45:18.102501 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:45:18.104218 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.084318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:46121)
0807 19:45:18.085446 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:18.085500 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:18.087472 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:18.088189 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:18.088198 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:18.088700 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:18.089351 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.089371 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.090446 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.090450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.091695 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.091708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.092673 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.092678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.092888 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.093577 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:18.093600 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:18.096027 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:18.099392 (+ 3365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:18.099401 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:18.099418 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:18.099977 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:18.100427 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:18.100431 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:18.100435 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:18.100670 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:18.101373 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:18.101381 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:18.101792 (+ 411us) client_negotiation.cc:769] Sending connection context
0807 19:45:18.102078 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 19:45:18.102605 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"thread_start_us":309,"threads_started":1}
I20260807 19:45:18.105849 15360 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33657
I20260807 19:45:18.106197 15360 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:18.107117 15360 heartbeater.cc:511] Master 127.30.208.62:33657 requested a full tablet report, sending...
I20260807 19:45:18.108925 15000 ts_manager.cc:194] Registered new tserver with Master: bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655)
I20260807 19:45:18.110090 15000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:46121
W20260807 19:45:18.426136 15364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:18.426456 15364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:18.426594 15364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:18.434036 15364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:18.434175 15364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:18.434333 15364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:45:18.444537 15364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:40887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=44205
--tserver_master_addrs=127.30.208.62:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:18.445943 15364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:18.447232 15364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:18.460253 15369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:18.460286 15370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:18.460328 15372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:18.461109 15364 server_base.cc:1034] running on GCE node
I20260807 19:45:18.461715 15364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:18.462668 15364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:18.463876 15364 hybrid_clock.cc:648] HybridClock initialized: now 1786131918463814 us; error 43 us; skew 500 ppm
I20260807 19:45:18.464178 15364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:18 dist-test-slave-vq2q krb5kdc[14354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:18.467453 15364 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:45:18.469128 15364 webserver.cc:460] Webserver started at http://127.30.208.3:44205/ using document root <none> and password file <none>
I20260807 19:45:18.469901 15364 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:18.470065 15364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:18.475975 15364 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 19:45:18.478878 15380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:18.480227 15364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:45:18.480391 15364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "43ea70e9351043a19cec0a410e071558"
format_stamp: "Formatted at 2026-08-07 19:45:15 on dist-test-slave-vq2q"
server_key: "e63f83b44c200eab4e13dce916cbb86b"
server_key_iv: "fe149cf6462f00f6a1ef29c1a31a55ce"
server_key_version: "encryptionkey@0"
I20260807 19:45:18.481436 15364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:18.499725 15364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:18.504077 15364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:18.504356 15364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:18.505465 15364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:18.506875 15364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:18.506979 15364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:18.507091 15364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:18.507169 15364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:18.540009 15364 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:40887
I20260807 19:45:18.540177 15511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:40887 every 8 connection(s)
I20260807 19:45:18.541544 15364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:45:18.546028 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 15364
Aug 07 19:45:18 dist-test-slave-vq2q krb5kdc[14354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:18.564965 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.544100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:52801 (local address 127.30.208.62:33657)
0807 19:45:18.544301 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:18.544308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:18.545485 (+ 1177us) server_negotiation.cc:407] Connection header received
0807 19:45:18.547424 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:18.547431 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:18.547527 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:18.547635 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:18.549870 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.550648 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.552267 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.552695 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.552846 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.557229 (+ 4383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:18.557235 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:18.557238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:18.557284 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:18.561365 (+ 4081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:18.562255 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:18.562259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:18.562262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:18.562427 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:18.562909 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:18.562913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:18.562915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:18.563207 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:18.563367 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:18.564291 (+ 924us) server_negotiation.cc:299] Negotiation successful
0807 19:45:18.564617 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":132,"server-negotiator.queue_time_us":58,"spinlock_wait_cycles":4864}
I20260807 19:45:18.565107 15515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.546904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53146 (local address 127.30.208.62:33657)
0807 19:45:18.547385 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:18.547390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:18.547984 (+ 594us) server_negotiation.cc:407] Connection header received
0807 19:45:18.548120 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:18.548123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:18.548194 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:18.548283 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:18.550259 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.551064 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.552489 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.553046 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.553168 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.557061 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:18.557067 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:18.557070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:18.557108 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:18.561421 (+ 4313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:18.562207 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:18.562213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:18.562216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:18.562427 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:18.563207 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:18.563212 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:18.563215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:18.563546 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:18.563741 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:18.564172 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 19:45:18.564482 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":366,"spinlock_wait_cycles":1920,"thread_start_us":240,"threads_started":1}
I20260807 19:45:18.566293 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.544578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:52801)
0807 19:45:18.545346 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:18.545384 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:18.547094 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:18.547922 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:18.547935 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:18.548673 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:18.549552 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.549582 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.550842 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.550846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.552120 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.552134 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.552948 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.552956 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.553279 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.554060 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:18.554099 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:18.557004 (+ 2905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:18.561589 (+ 4585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:18.561598 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:18.561614 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:18.562153 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:18.562564 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:18.562567 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:18.562570 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:18.562791 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:18.563359 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:18.563366 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:18.563856 (+ 490us) client_negotiation.cc:769] Sending connection context
0807 19:45:18.564198 (+ 342us) client_negotiation.cc:240] Negotiation successful
0807 19:45:18.564881 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":275,"threads_started":1}
I20260807 19:45:18.567790 15512 heartbeater.cc:347] Connected to a master server at 127.30.208.62:33657
I20260807 19:45:18.568089 15512 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:18.568888 15512 heartbeater.cc:511] Master 127.30.208.62:33657 requested a full tablet report, sending...
I20260807 19:45:18.570386 15000 ts_manager.cc:194] Registered new tserver with Master: 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887)
I20260807 19:45:18.571224 15000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:52801
I20260807 19:45:18.580217 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:45:18.588434 15211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.581271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53150 (local address 127.30.208.62:33657)
0807 19:45:18.581551 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:18.581555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:18.581964 (+ 409us) server_negotiation.cc:407] Connection header received
0807 19:45:18.582151 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:18.582155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:18.582307 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:18.582463 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:45:18.583061 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.583831 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.585739 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.586405 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.586543 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.587169 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:45:18.587849 (+ 680us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:45:18.588003 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:18.588062 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 19:45:18.588161 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260807 19:45:18.589609 15000 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:53150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:45:18.601236 15000 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:45:18.623477 15525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.613706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:40887 (local address 127.0.0.1:50166)
0807 19:45:18.614772 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:18.614788 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:18.614902 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:18.616003 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:18.616008 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:18.616034 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:18.616543 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.616556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.618552 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.618558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.620655 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.620672 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.622477 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.622483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.622732 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.622764 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:45:18.622856 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 19:45:18.623052 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"mutex_wait_us":91,"thread_start_us":822,"threads_started":1}
I20260807 19:45:18.623471 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.614292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50166 (local address 127.30.208.3:40887)
0807 19:45:18.615436 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:18.615444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:18.615476 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:45:18.615572 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:18.615579 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:18.615786 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:18.616568 (+ 782us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:18.616815 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.618393 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.620823 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.622221 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.622710 (+ 489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.622935 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:18.623015 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 19:45:18.623171 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":1155,"threads_started":1}
I20260807 19:45:18.623538 15524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.613251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53854 (local address 127.30.208.1:40919)
0807 19:45:18.614491 (+ 1240us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:18.614500 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:18.614528 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:45:18.614636 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:18.614643 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:18.614900 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:18.615700 (+ 800us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:18.615981 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.618067 (+ 2086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.620626 (+ 2559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.622104 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.622310 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.622554 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:18.623029 (+ 475us) server_negotiation.cc:299] Negotiation successful
0807 19:45:18.623195 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1081,"thread_start_us":925,"threads_started":1}
I20260807 19:45:18.624277 15209 heartbeater.cc:503] Master 127.30.208.62:33657 was elected leader, sending a full tablet report...
I20260807 19:45:18.624358 15523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.612918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:40919 (local address 127.0.0.1:53854)
0807 19:45:18.613645 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:18.613685 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:18.613889 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:18.615145 (+ 1256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:18.615153 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:18.615186 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:18.615585 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.615603 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.618222 (+ 2619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.618227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.620470 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.620483 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.622410 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.622417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.622620 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.622704 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 19:45:18.622844 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 19:45:18.623903 (+ 1059us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":962,"thread_start_us":528,"threads_started":1}
I20260807 19:45:18.624974 15528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.614728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43752 (local address 127.30.208.2:45655)
0807 19:45:18.616128 (+ 1400us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:18.616136 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:18.616894 (+ 758us) server_negotiation.cc:407] Connection header received
0807 19:45:18.617113 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:18.617121 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:18.617365 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:18.617527 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:18.618182 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.619486 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.621750 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.623166 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.623294 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.623477 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:18.624462 (+ 985us) server_negotiation.cc:299] Negotiation successful
0807 19:45:18.624644 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1252,"thread_start_us":1122,"threads_started":1}
I20260807 19:45:18.625398 15526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.614220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:45655 (local address 127.0.0.1:43752)
0807 19:45:18.616775 (+ 2555us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:18.616796 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:18.616950 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:18.617579 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:18.617585 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:18.617609 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:18.617950 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.617959 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.619631 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.619636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.621572 (+ 1936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.621584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.623733 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.623741 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.623940 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.623971 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 19:45:18.624064 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 19:45:18.624186 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2459,"mutex_wait_us":44,"thread_start_us":2108,"threads_started":1}
I20260807 19:45:18.628019 15435 tablet_service.cc:1467] Processing CreateTablet for tablet e5fc8a9b689c48c38b9d5539835df5dc (DEFAULT_TABLE table=test-table [id=9c2d95ba86604c4bad612c0ae96c928c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:18.629282 15435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5fc8a9b689c48c38b9d5539835df5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:18.628861 15285 tablet_service.cc:1467] Processing CreateTablet for tablet e5fc8a9b689c48c38b9d5539835df5dc (DEFAULT_TABLE table=test-table [id=9c2d95ba86604c4bad612c0ae96c928c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:18.628867 15135 tablet_service.cc:1467] Processing CreateTablet for tablet e5fc8a9b689c48c38b9d5539835df5dc (DEFAULT_TABLE table=test-table [id=9c2d95ba86604c4bad612c0ae96c928c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:18.630211 15285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5fc8a9b689c48c38b9d5539835df5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:18.630234 15135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5fc8a9b689c48c38b9d5539835df5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:18.642092 15529 tablet_bootstrap.cc:492] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558: Bootstrap starting.
I20260807 19:45:18.646410 15529 tablet_bootstrap.cc:654] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:18.647461 15529 log.cc:824] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:18.649837 15530 tablet_bootstrap.cc:492] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310: Bootstrap starting.
I20260807 19:45:18.650525 15529 tablet_bootstrap.cc:492] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558: No bootstrap required, opened a new log
I20260807 19:45:18.650771 15529 ts_tablet_manager.cc:1397] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:45:18.652704 15530 tablet_bootstrap.cc:654] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:18.652884 15531 tablet_bootstrap.cc:492] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e: Bootstrap starting.
I20260807 19:45:18.653888 15530 log.cc:824] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:18.656859 15531 tablet_bootstrap.cc:654] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:18.658203 15531 log.cc:824] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:18.659772 15530 tablet_bootstrap.cc:492] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310: No bootstrap required, opened a new log
I20260807 19:45:18.660058 15530 ts_tablet_manager.cc:1397] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 19:45:18.661341 15531 tablet_bootstrap.cc:492] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e: No bootstrap required, opened a new log
I20260807 19:45:18.661029 15529 raft_consensus.cc:348] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:18.661512 15529 raft_consensus.cc:374] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:18.661612 15531 ts_tablet_manager.cc:1397] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260807 19:45:18.661619 15529 raft_consensus.cc:729] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43ea70e9351043a19cec0a410e071558, State: Initialized, Role: FOLLOWER
I20260807 19:45:18.662274 15529 consensus_queue.cc:260] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:18.663445 15512 heartbeater.cc:503] Master 127.30.208.62:33657 was elected leader, sending a full tablet report...
I20260807 19:45:18.664481 15529 ts_tablet_manager.cc:1428] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260807 19:45:18.669328 15530 raft_consensus.cc:348] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:18.669713 15530 raft_consensus.cc:374] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:18.669816 15530 raft_consensus.cc:729] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c13a30ed4d64ff788a1214b92d2b310, State: Initialized, Role: FOLLOWER
I20260807 19:45:18.670444 15530 consensus_queue.cc:260] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:18.672211 15530 ts_tablet_manager.cc:1428] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260807 19:45:18.672400 15531 raft_consensus.cc:348] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:18.672796 15531 raft_consensus.cc:374] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:18.672905 15531 raft_consensus.cc:729] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf33209f9b2e4d76a3ca7fecebeeed1e, State: Initialized, Role: FOLLOWER
I20260807 19:45:18.673556 15531 consensus_queue.cc:260] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:18.674796 15360 heartbeater.cc:503] Master 127.30.208.62:33657 was elected leader, sending a full tablet report...
I20260807 19:45:18.675415 15531 ts_tablet_manager.cc:1428] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e: Time spent starting tablet: real 0.014s user 0.011s sys 0.000s
I20260807 19:45:18.744014 15536 raft_consensus.cc:482] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:18.744330 15536 raft_consensus.cc:504] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:18.746390 15536 leader_election.cc:288] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655), 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887)
I20260807 19:45:18.753230 15528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.747043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:59723 (local address 127.30.208.2:45655)
0807 19:45:18.747278 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:18.747285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:18.747721 (+ 436us) server_negotiation.cc:407] Connection header received
0807 19:45:18.748055 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:18.748076 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:18.748165 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:18.748255 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:18.749036 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.749793 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.751562 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.752337 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.752482 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.752559 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:18.752834 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 19:45:18.752959 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 19:45:18.753378 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.746590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:45655 (local address 127.30.208.1:59723)
0807 19:45:18.747593 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:18.747618 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:18.747877 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:18.748454 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:18.748459 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:18.748489 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:18.748899 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.748912 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.749919 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.749925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.751412 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.751424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.752535 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.752541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.752679 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.752703 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:45:18.752785 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 19:45:18.752914 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"mutex_wait_us":4,"thread_start_us":698,"threads_started":1}
I20260807 19:45:18.753808 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.747204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:46473 (local address 127.30.208.3:40887)
0807 19:45:18.747485 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:18.747505 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:18.748749 (+ 1244us) server_negotiation.cc:407] Connection header received
0807 19:45:18.748907 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:18.748912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:18.748997 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:18.749097 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:18.749660 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.750357 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.752051 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.752827 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.752973 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.753043 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:18.753447 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 19:45:18.753570 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 19:45:18.754458 15539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.747812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:40887 (local address 127.30.208.1:46473)
0807 19:45:18.748611 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:18.748635 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:18.748758 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:18.749127 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:18.749132 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:18.749158 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:18.749491 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.749501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.750492 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.750502 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.751903 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:18.751915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.753039 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.753045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:18.753215 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.753254 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 19:45:18.753975 (+ 721us) client_negotiation.cc:240] Negotiation successful
0807 19:45:18.754104 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":22,"thread_start_us":481,"threads_started":1}
I20260807 19:45:18.754628 15310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5fc8a9b689c48c38b9d5539835df5dc" candidate_uuid: "3c13a30ed4d64ff788a1214b92d2b310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" is_pre_election: true
I20260807 19:45:18.755149 15310 raft_consensus.cc:2454] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c13a30ed4d64ff788a1214b92d2b310 in term 0.
I20260807 19:45:18.755612 15459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5fc8a9b689c48c38b9d5539835df5dc" candidate_uuid: "3c13a30ed4d64ff788a1214b92d2b310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43ea70e9351043a19cec0a410e071558" is_pre_election: true
I20260807 19:45:18.755911 15078 leader_election.cc:302] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c13a30ed4d64ff788a1214b92d2b310, bf33209f9b2e4d76a3ca7fecebeeed1e; no voters:
I20260807 19:45:18.756148 15459 raft_consensus.cc:2454] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c13a30ed4d64ff788a1214b92d2b310 in term 0.
I20260807 19:45:18.756343 15536 raft_consensus.cc:2781] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:45:18.756500 15536 raft_consensus.cc:482] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:45:18.756610 15536 raft_consensus.cc:3037] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:18.758921 15536 raft_consensus.cc:504] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:18.759470 15536 leader_election.cc:288] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 1 election: Requested vote from peers bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655), 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887)
I20260807 19:45:18.759969 15459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5fc8a9b689c48c38b9d5539835df5dc" candidate_uuid: "3c13a30ed4d64ff788a1214b92d2b310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43ea70e9351043a19cec0a410e071558"
I20260807 19:45:18.759963 15310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5fc8a9b689c48c38b9d5539835df5dc" candidate_uuid: "3c13a30ed4d64ff788a1214b92d2b310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e"
I20260807 19:45:18.760180 15459 raft_consensus.cc:3037] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:18.760180 15310 raft_consensus.cc:3037] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:18.763469 15459 raft_consensus.cc:2454] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c13a30ed4d64ff788a1214b92d2b310 in term 1.
I20260807 19:45:18.763469 15310 raft_consensus.cc:2454] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c13a30ed4d64ff788a1214b92d2b310 in term 1.
I20260807 19:45:18.763971 15078 leader_election.cc:302] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c13a30ed4d64ff788a1214b92d2b310, bf33209f9b2e4d76a3ca7fecebeeed1e; no voters:
I20260807 19:45:18.764477 15536 raft_consensus.cc:2781] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:45:18.764976 15536 raft_consensus.cc:686] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 LEADER]: Becoming Leader. State: Replica: 3c13a30ed4d64ff788a1214b92d2b310, State: Running, Role: LEADER
I20260807 19:45:18.765540 15536 consensus_queue.cc:237] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:18.770097 14999 catalog_manager.cc:5495] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c13a30ed4d64ff788a1214b92d2b310 (127.30.208.1). New cstate: current_term: 1 leader_uuid: "3c13a30ed4d64ff788a1214b92d2b310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } health_report { overall_health: UNKNOWN } } }
W20260807 19:45:18.793212 15513 tablet.cc:2369] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:45:18.818492 15524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.813156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53858 (local address 127.30.208.1:40919)
0807 19:45:18.813360 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:18.813365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:18.813384 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:45:18.813578 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:18.813581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:18.813670 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:18.813788 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:45:18.814132 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.814845 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.816213 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.816652 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.816743 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.817275 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:45:18.817858 (+ 583us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:45:18.817975 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:18.818118 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 19:45:18.818207 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:45:18.827111 15459 raft_consensus.cc:1261] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558 [term 1 FOLLOWER]: Refusing update from remote peer 3c13a30ed4d64ff788a1214b92d2b310: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:45:18.827106 15310 raft_consensus.cc:1261] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e [term 1 FOLLOWER]: Refusing update from remote peer 3c13a30ed4d64ff788a1214b92d2b310: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:45:18.828111 15540 consensus_queue.cc:1035] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:45:18.828490 15536 consensus_queue.cc:1035] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:45:18.833293 15361 tablet.cc:2369] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:45:18.840055 15210 tablet.cc:2369] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:45:18.841682 15543 mvcc.cc:205] Tried to move back new op lower bound from 7315996339504857088 to 7315996339267907584. Current Snapshot: MvccSnapshot[applied={T|T < 7315996339504857088}]
I20260807 19:45:18.862365 14999 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:53150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:45:18.862661 14999 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:53150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:45:18.866176 14999 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Sending DeleteTablet for 3 replicas of tablet e5fc8a9b689c48c38b9d5539835df5dc
I20260807 19:45:18.867475 15285 tablet_service.cc:1514] Processing DeleteTablet for tablet e5fc8a9b689c48c38b9d5539835df5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:18 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:43752
I20260807 19:45:18.867789 15135 tablet_service.cc:1514] Processing DeleteTablet for tablet e5fc8a9b689c48c38b9d5539835df5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:18 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:53854
I20260807 19:45:18.867947 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 15038
I20260807 19:45:18.868330 15435 tablet_service.cc:1514] Processing DeleteTablet for tablet e5fc8a9b689c48c38b9d5539835df5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:18 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:50166
I20260807 19:45:18.877863 15555 tablet_replica.cc:330] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e: stopping tablet replica
I20260807 19:45:18.878451 15555 raft_consensus.cc:2229] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:45:18.879112 15555 raft_consensus.cc:2258] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:18.882087 15557 tablet_replica.cc:330] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558: stopping tablet replica
I20260807 19:45:18.882575 15557 raft_consensus.cc:2229] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:45:18.883123 15557 raft_consensus.cc:2258] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:18.885512 15557 ts_tablet_manager.cc:1905] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:45:18.892442 15556 tablet_replica.cc:330] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310: stopping tablet replica
I20260807 19:45:18.892894 15556 raft_consensus.cc:2229] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:45:18.893330 15524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:18.869577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53866 (local address 127.30.208.1:40919)
0807 19:45:18.869751 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:18.869761 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:18.869780 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:45:18.869847 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:18.869853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:18.869955 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:18.870054 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:18.872049 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.873280 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.875258 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:18.875984 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:18.876121 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:18.883885 (+ 7764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:18.883902 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:18.883920 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:18.883980 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:18.888407 (+ 4427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:18.889543 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:18.889553 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:18.889557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:18.889750 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:18.890253 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:18.890258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:18.890266 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:18.890753 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:18.891081 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:18.892482 (+ 1401us) server_negotiation.cc:299] Negotiation successful
0807 19:45:18.892731 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:45:18.893679 15556 raft_consensus.cc:2258] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:18.894295 15201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:18.894843 15555 ts_tablet_manager.cc:1905] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:45:18.895989 15557 ts_tablet_manager.cc:1918] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:45:18.896175 15557 log.cc:1196] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/e5fc8a9b689c48c38b9d5539835df5dc
I20260807 19:45:18.896754 15557 ts_tablet_manager.cc:1939] T e5fc8a9b689c48c38b9d5539835df5dc P 43ea70e9351043a19cec0a410e071558: Deleting consensus metadata
I20260807 19:45:18.898821 14986 catalog_manager.cc:4841] TS 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887): tablet e5fc8a9b689c48c38b9d5539835df5dc (table test-table [id=9c2d95ba86604c4bad612c0ae96c928c]) successfully deleted
I20260807 19:45:18.900883 15555 ts_tablet_manager.cc:1918] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:45:18.901028 15555 log.cc:1196] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/e5fc8a9b689c48c38b9d5539835df5dc
I20260807 19:45:18.901515 15555 ts_tablet_manager.cc:1939] T e5fc8a9b689c48c38b9d5539835df5dc P bf33209f9b2e4d76a3ca7fecebeeed1e: Deleting consensus metadata
I20260807 19:45:18.902985 14982 catalog_manager.cc:4841] TS bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655): tablet e5fc8a9b689c48c38b9d5539835df5dc (table test-table [id=9c2d95ba86604c4bad612c0ae96c928c]) successfully deleted
I20260807 19:45:18.992129 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 15038
W20260807 19:45:19.011390 14982 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.208.1:40919 (error 108)
I20260807 19:45:19.011545 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 15213
I20260807 19:45:19.013293 15526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:19.012827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:40919 (local address 127.0.0.1:53878)
0807 19:45:19.013011 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:19.013112 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:40919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 19:45:19.013686 14982 catalog_manager.cc:4584] TS 3c13a30ed4d64ff788a1214b92d2b310 (127.30.208.1:40919): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e5fc8a9b689c48c38b9d5539835df5dc: Network error: Client connection negotiation failed: client connection to 127.30.208.1:40919: connect: Connection refused (error 111)
I20260807 19:45:19.026374 15528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:19.012400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43760 (local address 127.30.208.2:45655)
0807 19:45:19.012602 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:19.012608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:19.012657 (+ 49us) server_negotiation.cc:407] Connection header received
0807 19:45:19.012717 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:19.012721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:19.012784 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:19.012859 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:19.014469 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:19.015392 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:19.016894 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:19.017475 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:19.017597 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:19.020383 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:19.020399 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:19.020415 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:19.020458 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:19.023509 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:19.024274 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:19.024284 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:19.024288 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:19.024423 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:19.024812 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:19.024815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:19.024817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:19.025242 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:19.025472 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:19.025874 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 19:45:19.026058 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:45:19.027046 15354 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:19.067580 15526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:19.067061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:40919 (local address 127.0.0.1:53890)
0807 19:45:19.067278 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:19.067390 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:40919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:45:19.068667 15526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:19.068267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:40919 (local address 127.0.0.1:53906)
0807 19:45:19.068442 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:19.068539 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:40919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 19:45:19.126819 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 15213
I20260807 19:45:19.129249 15526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:19.128647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:40919 (local address 127.0.0.1:53914)
0807 19:45:19.128900 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:19.129027 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:40919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 19:45:19.130501 15526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:19.130038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:40919 (local address 127.0.0.1:53916)
0807 19:45:19.130243 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:19.130359 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:40919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 19:45:19.144809 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 15364
I20260807 19:45:19.158480 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:19.145560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50172 (local address 127.30.208.3:40887)
0807 19:45:19.145776 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:19.145783 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:19.145806 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:45:19.145927 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:19.145932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:19.146011 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:19.146112 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:19.147482 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:19.148231 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:19.149628 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:19.150049 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:19.150156 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:19.152683 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:19.152699 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:19.152716 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:19.152752 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:19.155728 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:19.156456 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:19.156464 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:19.156466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:19.156610 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:19.157001 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:19.157004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:19.157006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:19.157417 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:19.157666 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:19.157954 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 19:45:19.158137 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:45:19.159173 15506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:19.211191 15526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:19.210566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:40919 (local address 127.0.0.1:53920)
0807 19:45:19.210834 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:19.210964 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:40919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 19:45:19.212399 15526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:19.211943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:40919 (local address 127.0.0.1:53928)
0807 19:45:19.212165 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:19.212254 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:40919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 19:45:19.248732 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 15364
I20260807 19:45:19.266633 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 14963
I20260807 19:45:19.267812 15031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:19.382629 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 14963
I20260807 19:45:19.405588 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33657
--webserver_interface=127.30.208.62
--webserver_port=34869
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:33657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:19.745344 15563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:19.745697 15563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:19.745842 15563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:19.754272 15563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:45:19.754413 15563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:19.754494 15563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:19.754576 15563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:45:19.754637 15563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:45:19.765146 15563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:33657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=34869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:19.766656 15563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:19.768044 15563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:19.780743 15569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:19.780943 15570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:19.781713 15572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:19.782485 15563 server_base.cc:1034] running on GCE node
I20260807 19:45:19.783383 15563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:19.784883 15563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:19.786103 15563 hybrid_clock.cc:648] HybridClock initialized: now 1786131919786056 us; error 32 us; skew 500 ppm
I20260807 19:45:19.786460 15563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:19 dist-test-slave-vq2q krb5kdc[14354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131919, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:19.789669 15563 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:45:19.791218 15563 webserver.cc:460] Webserver started at http://127.30.208.62:34869/ using document root <none> and password file <none>
I20260807 19:45:19.791934 15563 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:19.792058 15563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:19.798650 15563 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:45:19.802307 15580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:19.803987 15563 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 19:45:19.804186 15563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "f9bc628fe0ca487b9b9b54b4a34c3343"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-vq2q"
server_key: "9b5e8bcfb16fb1c99e629dbb3b3808ba"
server_key_iv: "69729eb0354c04256e2242de9422fd14"
server_key_version: "encryptionkey@0"
I20260807 19:45:19.805289 15563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:19.829995 15563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:19.834275 15563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:19.834544 15563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:19.853145 15563 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:33657
I20260807 19:45:19.853154 15634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:33657 every 8 connection(s)
I20260807 19:45:19.854717 15563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:45:19.861843 15635 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:45:19.862887 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 15563
I20260807 19:45:19.863705 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:40919
--local_ip_for_outbound_sockets=127.30.208.1
--tserver_master_addrs=127.30.208.62:33657
--webserver_port=35067
--webserver_interface=127.30.208.1
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:45:19.865384 15635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Bootstrap starting.
I20260807 19:45:19.890422 15635 log.cc:824] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:19.912281 15635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:45:19.912976 15635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343: Bootstrap complete.
I20260807 19:45:19.926959 15635 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } }
I20260807 19:45:19.928369 15635 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9bc628fe0ca487b9b9b54b4a34c3343, State: Initialized, Role: FOLLOWER
I20260807 19:45:19.929311 15635 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } }
I20260807 19:45:19.929616 15635 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:45:19.929750 15635 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:45:19.929950 15635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:45:19.935783 15635 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } }
I20260807 19:45:19.936414 15635 leader_election.cc:302] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9bc628fe0ca487b9b9b54b4a34c3343; no voters:
I20260807 19:45:19.937191 15635 leader_election.cc:288] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 19:45:19.937639 15640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:45:19.938840 15640 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [term 3 LEADER]: Becoming Leader. State: Replica: f9bc628fe0ca487b9b9b54b4a34c3343, State: Running, Role: LEADER
I20260807 19:45:19.939383 15640 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } }
I20260807 19:45:19.940153 15635 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:45:19.942258 15642 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9bc628fe0ca487b9b9b54b4a34c3343. Latest consensus state: current_term: 3 leader_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } } }
I20260807 19:45:19.942716 15642 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:19.942004 15641 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bc628fe0ca487b9b9b54b4a34c3343" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33657 } } }
I20260807 19:45:19.943351 15641 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9bc628fe0ca487b9b9b54b4a34c3343 [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:19.945443 15648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:45:19.954460 15648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:45:19.956099 15648 catalog_manager.cc:1259] Loaded cluster ID: ed858bbd30a94d318c6f9748023128ba
I20260807 19:45:19.956781 15648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 19:45:19.959007 15648 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f90df31b9ad google::LogMessage::Fail() at ??:0
@ 0x7f90df31e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f90df31b4de google::LogMessage::Flush() at ??:0
@ 0x7f90df31ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f90f4d4e512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f90f4d4b976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f90f4dd3610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f90f4dd3439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f90e1f67345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f90e1f6a91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f90e1f6a799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f90e1f45eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f90e521f6db start_thread at ??:0
@ 0x7f90de1b561f clone at ??:0
W20260807 19:45:20.275101 15637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:20.275458 15637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:20.275604 15637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:20.283664 15637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:20.283842 15637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:20.284026 15637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:45:20.295488 15637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:40919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=35067
--tserver_master_addrs=127.30.208.62:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:20.296931 15637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:20.298404 15637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:20.314567 15684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:20.314487 15687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:20.314529 15685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:20.315508 15637 server_base.cc:1034] running on GCE node
I20260807 19:45:20.316103 15637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:20.317106 15637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:20.318369 15637 hybrid_clock.cc:648] HybridClock initialized: now 1786131920318318 us; error 41 us; skew 500 ppm
I20260807 19:45:20.318660 15637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:20 dist-test-slave-vq2q krb5kdc[14354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131920, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:20.321853 15637 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:45:20.323540 15637 webserver.cc:460] Webserver started at http://127.30.208.1:35067/ using document root <none> and password file <none>
I20260807 19:45:20.324234 15637 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:20.324355 15637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:20.330526 15637 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 19:45:20.333854 15696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:20.335307 15637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:45:20.335469 15637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "3c13a30ed4d64ff788a1214b92d2b310"
format_stamp: "Formatted at 2026-08-07 19:45:14 on dist-test-slave-vq2q"
server_key: "598062ce90302438445bf5aaec791433"
server_key_iv: "aca159469318ee7f5a4723f881401598"
server_key_version: "encryptionkey@0"
I20260807 19:45:20.336457 15637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:20.358186 15637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:20.362632 15637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:20.362895 15637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:20.364116 15637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:20.366387 15703 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 19:45:20.372298 15637 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 19:45:20.372437 15637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260807 19:45:20.372567 15637 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 19:45:20.375828 15637 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 19:45:20.375914 15637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260807 19:45:20.376323 15703 tablet_bootstrap.cc:492] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310: Bootstrap starting.
I20260807 19:45:20.384687 15703 log.cc:824] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:20.401559 15703 tablet_bootstrap.cc:492] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:45:20.402539 15703 tablet_bootstrap.cc:492] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310: Bootstrap complete.
I20260807 19:45:20.403160 15703 ts_tablet_manager.cc:1397] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310: Time spent bootstrapping tablet: real 0.027s user 0.019s sys 0.005s
I20260807 19:45:20.412703 15703 raft_consensus.cc:348] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:20.414294 15703 raft_consensus.cc:729] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c13a30ed4d64ff788a1214b92d2b310, State: Initialized, Role: FOLLOWER
I20260807 19:45:20.415479 15703 consensus_queue.cc:260] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:20.417373 15703 ts_tablet_manager.cc:1428] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
I20260807 19:45:20.417734 15637 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:40919
I20260807 19:45:20.417757 15823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:40919 every 8 connection(s)
I20260807 19:45:20.419044 15637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:45:20.420702 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 15637
I20260807 19:45:20.421500 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:45655
--local_ip_for_outbound_sockets=127.30.208.2
--tserver_master_addrs=127.30.208.62:33657
--webserver_port=40531
--webserver_interface=127.30.208.2
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:45:20.433171 15827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.421746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:44017)
0807 19:45:20.431665 (+ 9919us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:20.432038 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9536,"thread_start_us":9281,"threads_started":1}
W20260807 19:45:20.433902 15824 heartbeater.cc:644] Failed to heartbeat to 127.30.208.62:33657 (0 consecutive failures): Network error: Failed to ping master at 127.30.208.62:33657: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
I20260807 19:45:20.434852 15827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.434504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:39817)
0807 19:45:20.434666 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:20.434740 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 19:45:20.435719 15827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.435372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:37743)
0807 19:45:20.435526 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:20.435600 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 19:45:20.435950 15824 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:45:20.775411 15828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:20.775738 15828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:20.775870 15828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:20.783347 15828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:20.783505 15828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:20.783668 15828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:45:20.793998 15828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:45655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=40531
--tserver_master_addrs=127.30.208.62:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:20.795337 15828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:20.796808 15828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:20.810264 15838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:20.810329 15839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:20.810468 15841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:20.811206 15828 server_base.cc:1034] running on GCE node
I20260807 19:45:20.811859 15828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:20.812866 15828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:20.814106 15828 hybrid_clock.cc:648] HybridClock initialized: now 1786131920814047 us; error 46 us; skew 500 ppm
I20260807 19:45:20.814451 15828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:20 dist-test-slave-vq2q krb5kdc[14354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131920, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:20.818222 15828 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:45:20.819825 15828 webserver.cc:460] Webserver started at http://127.30.208.2:40531/ using document root <none> and password file <none>
I20260807 19:45:20.820600 15828 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:20.820713 15828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:20.826542 15828 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 19:45:20.829778 15848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:20.831260 15828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:45:20.831456 15828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e"
format_stamp: "Formatted at 2026-08-07 19:45:15 on dist-test-slave-vq2q"
server_key: "7b79565c778fbcc07f52eb1c18cdec94"
server_key_iv: "c881a631fe427dae4ae41d4b69bf2dfe"
server_key_version: "encryptionkey@0"
I20260807 19:45:20.832525 15828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:20.852718 15828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:20.857671 15828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:20.857976 15828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:20.859114 15828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:20.860883 15828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:20.860996 15828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:20.861130 15828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:20.861225 15828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:20.895391 15828 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:45655
I20260807 19:45:20.895442 15974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:45655 every 8 connection(s)
I20260807 19:45:20.897111 15828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:45:20.902060 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 15828
I20260807 19:45:20.902266 15977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.900232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:36699)
0807 19:45:20.901059 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:20.901326 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":296,"threads_started":1}
I20260807 19:45:20.902870 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:40887
--local_ip_for_outbound_sockets=127.30.208.3
--tserver_master_addrs=127.30.208.62:33657
--webserver_port=44205
--webserver_interface=127.30.208.3
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:45687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:20.903057 15975 heartbeater.cc:644] Failed to heartbeat to 127.30.208.62:33657 (0 consecutive failures): Network error: Failed to ping master at 127.30.208.62:33657: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
I20260807 19:45:20.904022 15977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.903641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:58443)
0807 19:45:20.903834 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:20.903910 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 19:45:20.904887 15977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:20.904555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:37641)
0807 19:45:20.904694 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:20.904778 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 19:45:20.905119 15975 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 19:45:21.252286 15978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:21.252563 15978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:21.252658 15978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:21.259953 15978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:21.260071 15978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:21.260187 15978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:45:21.270092 15978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:40887
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=44205
--tserver_master_addrs=127.30.208.62:33657
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:21.271358 15978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:21.272676 15978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:21.287635 15985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:21.287598 15986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:21.287827 15988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:21.289070 15978 server_base.cc:1034] running on GCE node
I20260807 19:45:21.289877 15978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:21.290977 15978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:21.292209 15978 hybrid_clock.cc:648] HybridClock initialized: now 1786131921292141 us; error 64 us; skew 500 ppm
I20260807 19:45:21.292852 15978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:21 dist-test-slave-vq2q krb5kdc[14354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131921, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:21.297257 15978 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:45:21.299435 15978 webserver.cc:460] Webserver started at http://127.30.208.3:44205/ using document root <none> and password file <none>
I20260807 19:45:21.300657 15978 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:21.300853 15978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:21.307279 15978 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:45:21.310869 15998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:21.313086 15978 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 19:45:21.313436 15978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "43ea70e9351043a19cec0a410e071558"
format_stamp: "Formatted at 2026-08-07 19:45:15 on dist-test-slave-vq2q"
server_key: "e63f83b44c200eab4e13dce916cbb86b"
server_key_iv: "fe149cf6462f00f6a1ef29c1a31a55ce"
server_key_version: "encryptionkey@0"
I20260807 19:45:21.315526 15978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:21.336098 15978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:21.343573 15978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:21.343891 15978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:21.345307 15978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:21.346932 15978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:21.347050 15978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:21.347168 15978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:21.347246 15978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:21.383380 15978 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:40887
I20260807 19:45:21.383445 16128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:40887 every 8 connection(s)
I20260807 19:45:21.384989 15978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:45:21.390816 16131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:21.388251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:47759)
0807 19:45:21.389111 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:21.389393 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":319,"threads_started":1}
W20260807 19:45:21.392108 16129 heartbeater.cc:644] Failed to heartbeat to 127.30.208.62:33657 (0 consecutive failures): Network error: Failed to ping master at 127.30.208.62:33657: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
I20260807 19:45:21.393430 16131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:21.392739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:46829)
0807 19:45:21.393039 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:21.393169 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 19:45:21.394119 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 15978
I20260807 19:45:21.394740 16131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:21.394237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:35183)
0807 19:45:21.394452 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:21.394558 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 19:45:21.395136 16129 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 19:45:21.437498 16133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:21.436551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:47825)
0807 19:45:21.437113 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:21.437286 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":268,"threads_started":1}
I20260807 19:45:21.906666 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:21.905842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:39355)
0807 19:45:21.906316 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:21.906434 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":228,"threads_started":1}
I20260807 19:45:22.036275 16135 raft_consensus.cc:482] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:22.036630 16135 raft_consensus.cc:504] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:22.038242 16135 leader_election.cc:288] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655), 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887)
Aug 07 19:45:22 dist-test-slave-vq2q krb5kdc[14354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131920, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
Aug 07 19:45:22 dist-test-slave-vq2q krb5kdc[14354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131920, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:45:22.060619 16138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:22.038899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:36511 (local address 127.30.208.2:45655)
0807 19:45:22.040678 (+ 1779us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:22.040692 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:22.040741 (+ 49us) server_negotiation.cc:407] Connection header received
0807 19:45:22.042976 (+ 2235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:22.042985 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:22.043479 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:22.043855 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:22.045671 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:22.046936 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:22.048723 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:22.049403 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:22.049587 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:22.053634 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:22.053646 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:22.053666 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:22.053721 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:22.056538 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:22.057493 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:22.057503 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:22.057510 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:22.057687 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:22.058203 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:22.058208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:22.058211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:22.058848 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:22.059019 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:22.059863 (+ 844us) server_negotiation.cc:299] Negotiation successful
0807 19:45:22.060213 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1593,"thread_start_us":1428,"threads_started":1}
I20260807 19:45:22.060624 16139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:22.039295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:33053 (local address 127.30.208.3:40887)
0807 19:45:22.040833 (+ 1538us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:22.040845 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:22.040877 (+ 32us) server_negotiation.cc:407] Connection header received
0807 19:45:22.042944 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:22.042958 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:22.043479 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:22.043855 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:22.045708 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:22.046886 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:22.048727 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:22.049405 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:22.049573 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:22.053916 (+ 4343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:22.053925 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:22.053935 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:22.053970 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:22.056761 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:22.057579 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:22.057585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:22.057591 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:22.057824 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:22.058341 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:22.058345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:22.058347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:22.058844 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:22.059063 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:22.059826 (+ 763us) server_negotiation.cc:299] Negotiation successful
0807 19:45:22.060216 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1140,"thread_start_us":1230,"threads_started":1}
I20260807 19:45:22.060647 16137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:22.038586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:45655 (local address 127.30.208.1:36511)
0807 19:45:22.039262 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:22.039315 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:22.039952 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:22.043982 (+ 4030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:22.043990 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:22.044705 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:22.045435 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:22.045459 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:22.047104 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:22.047108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:22.048584 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:22.048598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:22.049653 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:22.049661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:22.049887 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:22.050662 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:22.050696 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:22.053311 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:22.056701 (+ 3390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:22.056710 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:22.056725 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:22.057345 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:22.057821 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:22.057826 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:22.057830 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:22.058056 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:22.058996 (+ 940us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:22.059004 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:22.059324 (+ 320us) client_negotiation.cc:769] Sending connection context
0807 19:45:22.059575 (+ 251us) client_negotiation.cc:240] Negotiation successful
0807 19:45:22.060146 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":144,"spinlock_wait_cycles":45696,"thread_start_us":698,"threads_started":1}
I20260807 19:45:22.060714 16136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:22.038586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:40887 (local address 127.30.208.1:33053)
0807 19:45:22.039630 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:22.039649 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:22.039941 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:22.044441 (+ 4500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:22.044445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:22.044945 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:22.045505 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:22.045522 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:22.047108 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:22.047112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:22.048584 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:22.048598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:22.049624 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:22.049631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:22.049886 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:22.050670 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:22.050694 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:22.053654 (+ 2960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:22.056901 (+ 3247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:22.056904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:22.056907 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:22.057417 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:22.057973 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:22.057977 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:22.057979 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:22.058172 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:22.059010 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:22.059015 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:22.059324 (+ 309us) client_negotiation.cc:769] Sending connection context
0807 19:45:22.059575 (+ 251us) client_negotiation.cc:240] Negotiation successful
0807 19:45:22.060305 (+ 730us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"mutex_wait_us":472,"spinlock_wait_cycles":45824,"thread_start_us":278,"threads_started":1}
W20260807 19:45:22.063961 15697 leader_election.cc:341] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655): Not found: Tablet not found: e5fc8a9b689c48c38b9d5539835df5dc
W20260807 19:45:22.064440 15700 leader_election.cc:341] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887): Not found: Tablet not found: e5fc8a9b689c48c38b9d5539835df5dc
I20260807 19:45:22.064589 15700 leader_election.cc:302] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c13a30ed4d64ff788a1214b92d2b310; no voters: 43ea70e9351043a19cec0a410e071558, bf33209f9b2e4d76a3ca7fecebeeed1e
I20260807 19:45:22.064998 16135 raft_consensus.cc:2726] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:45:22.396708 16140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:22.395805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:50471)
0807 19:45:22.396307 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:22.396461 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":236,"threads_started":1}
I20260807 19:45:22.438936 16136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:22.438466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:40585)
0807 19:45:22.438675 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:22.438780 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 19:45:22.908209 16141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:22.907530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:52239)
0807 19:45:22.907969 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:22.908050 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":240,"threads_started":1}
I20260807 19:45:23.398492 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:23.397785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:33439)
0807 19:45:23.398215 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:23.398327 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":220,"threads_started":1}
I20260807 19:45:23.440680 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:23.439887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:58149)
0807 19:45:23.440349 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:23.440450 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":224,"threads_started":1}
I20260807 19:45:23.851568 16144 raft_consensus.cc:482] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:23.851788 16144 raft_consensus.cc:504] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:23.852487 16144 leader_election.cc:288] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655), 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887)
W20260807 19:45:23.853562 15700 leader_election.cc:341] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887): Not found: Tablet not found: e5fc8a9b689c48c38b9d5539835df5dc
W20260807 19:45:23.853713 15697 leader_election.cc:341] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655): Not found: Tablet not found: e5fc8a9b689c48c38b9d5539835df5dc
I20260807 19:45:23.853874 15697 leader_election.cc:302] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c13a30ed4d64ff788a1214b92d2b310; no voters: 43ea70e9351043a19cec0a410e071558, bf33209f9b2e4d76a3ca7fecebeeed1e
I20260807 19:45:23.854164 16144 raft_consensus.cc:2726] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:45:23.910100 16145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:23.909290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:50895)
0807 19:45:23.909822 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:23.909919 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":286,"threads_started":1}
I20260807 19:45:24.400506 16146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:24.399694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:50303)
0807 19:45:24.400241 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:24.400327 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":278,"threads_started":1}
I20260807 19:45:24.442440 16147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:24.441707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:54849)
0807 19:45:24.442172 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:24.442267 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":238,"threads_started":1}
I20260807 19:45:24.911671 16148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:24.910975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:53323)
0807 19:45:24.911401 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:24.911485 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":206,"threads_started":1}
I20260807 19:45:25.402266 16149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:25.401513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:58379)
0807 19:45:25.401986 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:25.402076 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":235,"threads_started":1}
I20260807 19:45:25.444275 16150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:25.443469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:38071)
0807 19:45:25.443977 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:25.444084 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":236,"threads_started":1}
I20260807 19:45:25.913789 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:25.912864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:57701)
0807 19:45:25.913493 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:25.913606 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":354,"threads_started":1}
I20260807 19:45:26.404201 16153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:26.403426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:50749)
0807 19:45:26.403943 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:26.404029 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":294,"threads_started":1}
I20260807 19:45:26.446107 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:26.445313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:58555)
0807 19:45:26.445847 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:26.445939 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":269,"threads_started":1}
I20260807 19:45:26.584576 16155 raft_consensus.cc:482] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:26.584769 16155 raft_consensus.cc:504] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:26.585522 16155 leader_election.cc:288] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655), 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887)
W20260807 19:45:26.586551 15700 leader_election.cc:341] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887): Not found: Tablet not found: e5fc8a9b689c48c38b9d5539835df5dc
W20260807 19:45:26.586748 15697 leader_election.cc:341] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655): Not found: Tablet not found: e5fc8a9b689c48c38b9d5539835df5dc
I20260807 19:45:26.586899 15697 leader_election.cc:302] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c13a30ed4d64ff788a1214b92d2b310; no voters: 43ea70e9351043a19cec0a410e071558, bf33209f9b2e4d76a3ca7fecebeeed1e
I20260807 19:45:26.587188 16155 raft_consensus.cc:2726] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:45:26.915589 16156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:26.914789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:47109)
0807 19:45:26.915298 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:26.915392 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":255,"threads_started":1}
I20260807 19:45:27.405967 16157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:27.405141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:59685)
0807 19:45:27.405676 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:27.405759 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":270,"threads_started":1}
I20260807 19:45:27.447969 16158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:27.447112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:52641)
0807 19:45:27.447665 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:27.447773 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":321,"threads_started":1}
I20260807 19:45:27.917696 16159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:27.916722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:56305)
0807 19:45:27.917370 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:27.917471 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":269,"threads_started":1}
I20260807 19:45:28.407773 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:28.406940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:45195)
0807 19:45:28.407485 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:28.407567 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":284,"threads_started":1}
I20260807 19:45:28.449936 16161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:28.449019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:40923)
0807 19:45:28.449643 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:28.449742 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":389,"threads_started":1}
I20260807 19:45:28.919411 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:28.918625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:57715)
0807 19:45:28.919137 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:28.919225 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":299,"threads_started":1}
I20260807 19:45:29.409685 16163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:29.408855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:36269)
0807 19:45:29.409384 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:29.409488 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":245,"threads_started":1}
I20260807 19:45:29.451663 16164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:29.450916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:49943)
0807 19:45:29.451380 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:29.451483 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":239,"threads_started":1}
I20260807 19:45:29.844626 16166 raft_consensus.cc:482] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:29.844887 16166 raft_consensus.cc:504] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:29.845578 16166 leader_election.cc:288] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655), 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887)
W20260807 19:45:29.846688 15700 leader_election.cc:341] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887): Not found: Tablet not found: e5fc8a9b689c48c38b9d5539835df5dc
W20260807 19:45:29.846858 15697 leader_election.cc:341] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655): Not found: Tablet not found: e5fc8a9b689c48c38b9d5539835df5dc
I20260807 19:45:29.846974 15697 leader_election.cc:302] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c13a30ed4d64ff788a1214b92d2b310; no voters: 43ea70e9351043a19cec0a410e071558, bf33209f9b2e4d76a3ca7fecebeeed1e
I20260807 19:45:29.847214 16166 raft_consensus.cc:2726] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:45:29.921226 16167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:29.920377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:40985)
0807 19:45:29.920898 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:29.921001 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":250,"threads_started":1}
I20260807 19:45:30.411362 16168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:30.410632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:42319)
0807 19:45:30.411109 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:30.411189 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":223,"threads_started":1}
I20260807 19:45:30.453428 16169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:30.452692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:47537)
0807 19:45:30.453128 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:30.453254 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":228,"threads_started":1}
I20260807 19:45:30.923007 16170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:30.922262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:40175)
0807 19:45:30.922732 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:30.922827 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":248,"threads_started":1}
I20260807 19:45:31.413057 16171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:31.412343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:50095)
0807 19:45:31.412776 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:31.412874 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":234,"threads_started":1}
I20260807 19:45:31.455082 16172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:31.454329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:36337)
0807 19:45:31.454797 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:31.454901 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":238,"threads_started":1}
I20260807 19:45:31.924817 16173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:31.924031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:57143)
0807 19:45:31.924486 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:31.924597 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":198,"threads_started":1}
I20260807 19:45:32.414992 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:32.414157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:46247)
0807 19:45:32.414695 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:32.414826 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":250,"threads_started":1}
I20260807 19:45:32.456735 16175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:32.455982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:44093)
0807 19:45:32.456480 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:32.456574 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":253,"threads_started":1}
I20260807 19:45:32.926734 16176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:32.925916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:57105)
0807 19:45:32.926426 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:32.926530 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":266,"threads_started":1}
I20260807 19:45:33.416715 16177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:33.415972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:58001)
0807 19:45:33.416461 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:33.416544 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":260,"threads_started":1}
I20260807 19:45:33.458572 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:33.457767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:42227)
0807 19:45:33.458298 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:33.458386 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":277,"threads_started":1}
I20260807 19:45:33.928561 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:33.927799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:40399)
0807 19:45:33.928292 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:33.928391 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":250,"threads_started":1}
I20260807 19:45:34.418776 16184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:34.417862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:53673)
0807 19:45:34.418447 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:34.418564 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":329,"threads_started":1}
I20260807 19:45:34.460362 16185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:34.459554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:45113)
0807 19:45:34.460061 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:34.460155 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":275,"threads_started":1}
I20260807 19:45:34.857856 16186 raft_consensus.cc:482] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:34.858067 16186 raft_consensus.cc:504] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c13a30ed4d64ff788a1214b92d2b310" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 40919 } } peers { permanent_uuid: "bf33209f9b2e4d76a3ca7fecebeeed1e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 45655 } } peers { permanent_uuid: "43ea70e9351043a19cec0a410e071558" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40887 } }
I20260807 19:45:34.858814 16186 leader_election.cc:288] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655), 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887)
W20260807 19:45:34.859701 15697 leader_election.cc:341] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bf33209f9b2e4d76a3ca7fecebeeed1e (127.30.208.2:45655): Not found: Tablet not found: e5fc8a9b689c48c38b9d5539835df5dc
W20260807 19:45:34.859866 15700 leader_election.cc:341] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43ea70e9351043a19cec0a410e071558 (127.30.208.3:40887): Not found: Tablet not found: e5fc8a9b689c48c38b9d5539835df5dc
I20260807 19:45:34.859989 15700 leader_election.cc:302] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c13a30ed4d64ff788a1214b92d2b310; no voters: 43ea70e9351043a19cec0a410e071558, bf33209f9b2e4d76a3ca7fecebeeed1e
I20260807 19:45:34.860252 16186 raft_consensus.cc:2726] T e5fc8a9b689c48c38b9d5539835df5dc P 3c13a30ed4d64ff788a1214b92d2b310 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:45:34.930382 16187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:34.929604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:34969)
0807 19:45:34.930075 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:34.930189 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":229,"threads_started":1}
I20260807 19:45:35.420508 16188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:35.419760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:51455)
0807 19:45:35.420223 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:35.420316 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":240,"threads_started":1}
I20260807 19:45:35.462401 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:35.461633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.1:46857)
0807 19:45:35.462143 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:35.462230 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":285,"threads_started":1}
I20260807 19:45:35.932368 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:35.931553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.2:45017)
0807 19:45:35.932078 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:35.932210 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":261,"threads_started":1}
I20260807 19:45:36.404062 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 15637
I20260807 19:45:36.420048 16192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:36.405094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47000 (local address 127.30.208.1:40919)
0807 19:45:36.405628 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:36.405635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:36.405682 (+ 47us) server_negotiation.cc:407] Connection header received
0807 19:45:36.405763 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:36.405770 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:36.406056 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:36.406194 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:36.408480 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:36.409546 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:36.410842 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:36.411424 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:36.411528 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:36.414342 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:36.414354 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:36.414369 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:36.414402 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:36.417162 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:36.417929 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:36.417937 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:36.417939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:36.418075 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:36.418428 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:36.418434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:36.418437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:36.418860 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:36.419165 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:36.419514 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 19:45:36.419733 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":253,"threads_started":1}
I20260807 19:45:36.421608 15817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:36.422303 16194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:36.421534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:33657 (local address 127.30.208.3:54161)
0807 19:45:36.422001 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:36.422132 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:33657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":241,"threads_started":1}
I20260807 19:45:36.510000 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 15637
I20260807 19:45:36.528317 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 15828
I20260807 19:45:36.541625 16197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:36.529072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41956 (local address 127.30.208.2:45655)
0807 19:45:36.529567 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:36.529572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:36.529589 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:45:36.529644 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:36.529648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:36.529706 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:36.529799 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:36.531126 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:36.532006 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:36.533310 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:36.533739 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:36.533832 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:36.536417 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:36.536423 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:36.536426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:36.536461 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:36.539165 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:36.539833 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:36.539838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:36.539840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:36.539970 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:36.540265 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:36.540268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:36.540270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:36.540526 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:36.540729 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:36.541046 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 19:45:36.541265 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":267,"threads_started":1}
I20260807 19:45:36.542325 15970 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:36.627322 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 15828
I20260807 19:45:36.645357 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 15978
I20260807 19:45:36.658862 16199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:36.646228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43946 (local address 127.30.208.3:40887)
0807 19:45:36.646786 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:36.646791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:36.646808 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:45:36.646861 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:36.646864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:36.646954 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:36.647042 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:36.648409 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:36.649248 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:36.650527 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:36.650937 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:36.651026 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:36.653847 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:36.653852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:36.653855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:36.653888 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:36.656493 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:36.657166 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:36.657169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:36.657172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:36.657362 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:36.657730 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:36.657733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:36.657735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:36.657997 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:36.658203 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:36.658414 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 19:45:36.658576 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":284,"threads_started":1}
I20260807 19:45:36.659556 16124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:36.742959 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 15978
2026-08-07T19:45:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24465 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:45:36 dist-test-slave-vq2q krb5kdc[16205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:45:36 dist-test-slave-vq2q krb5kdc[16205](info): set up 2 sockets
Aug 07 19:45:36 dist-test-slave-vq2q krb5kdc[16205](info): commencing operation
krb5kdc: starting...
W20260807 19:45:37.826480 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:45:38 dist-test-slave-vq2q krb5kdc[16205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:45:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:45:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:45:38.109512 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:37421
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:37421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:38.447594 16222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:38.447885 16222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:38.447974 16222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:38.455140 16222 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 19:45:38.455250 16222 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 19:45:38.455286 16222 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 19:45:38.455338 16222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:45:38.455390 16222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:38.455456 16222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:38.455500 16222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:45:38.455552 16222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:45:38.465207 16222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:37421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:37421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:38.466434 16222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:38.467720 16222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:38.479494 16227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:38.479743 16222 server_base.cc:1034] running on GCE node
W20260807 19:45:38.479635 16229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:38.479662 16232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:38.480827 16222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:38.482309 16222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:38.483582 16222 hybrid_clock.cc:648] HybridClock initialized: now 1786131938483495 us; error 36 us; skew 500 ppm
I20260807 19:45:38.483955 16222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:38 dist-test-slave-vq2q krb5kdc[16205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131938, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:38.490526 16222 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:45:38.492185 16222 webserver.cc:460] Webserver started at http://127.30.208.62:40599/ using document root <none> and password file <none>
I20260807 19:45:38.492918 16222 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:38.493014 16222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:38.493381 16222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:38.496487 16222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "13947c19c459464cb19d9bf29abdf38d"
format_stamp: "Formatted at 2026-08-07 19:45:38 on dist-test-slave-vq2q"
server_key: "2cb9b4ea3254fe45dd34d9bd1e56471e"
server_key_iv: "854738b13ad05a167f7a5ca5bcbf0dcc"
server_key_version: "encryptionkey@0"
I20260807 19:45:38.497352 16222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "13947c19c459464cb19d9bf29abdf38d"
format_stamp: "Formatted at 2026-08-07 19:45:38 on dist-test-slave-vq2q"
server_key: "2cb9b4ea3254fe45dd34d9bd1e56471e"
server_key_iv: "854738b13ad05a167f7a5ca5bcbf0dcc"
server_key_version: "encryptionkey@0"
I20260807 19:45:38.502784 16222 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:45:38.506807 16240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:38.508406 16222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:45:38.508669 16222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "13947c19c459464cb19d9bf29abdf38d"
format_stamp: "Formatted at 2026-08-07 19:45:38 on dist-test-slave-vq2q"
server_key: "2cb9b4ea3254fe45dd34d9bd1e56471e"
server_key_iv: "854738b13ad05a167f7a5ca5bcbf0dcc"
server_key_version: "encryptionkey@0"
I20260807 19:45:38.508934 16222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:38.557550 16222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:38.562090 16222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:38.562362 16222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:38.580798 16222 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:37421
I20260807 19:45:38.580845 16298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:37421 every 8 connection(s)
I20260807 19:45:38.582235 16222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:45:38.586174 16299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:38.587641 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 16222
I20260807 19:45:38.587853 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:45:38.588527 31552 external_mini_cluster.cc:1428] Setting key 06939ec0187ed46ff71ef397347c6d34
I20260807 19:45:38.600448 16299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Bootstrap starting.
Aug 07 19:45:38 dist-test-slave-vq2q krb5kdc[16205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:38.603572 16299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:38.604632 16299 log.cc:824] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:38.608145 16299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: No bootstrap required, opened a new log
I20260807 19:45:38.613427 16302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:38.590586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50230 (local address 127.30.208.62:37421)
0807 19:45:38.591437 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:38.591453 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:38.591536 (+ 83us) server_negotiation.cc:407] Connection header received
0807 19:45:38.593333 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:38.593365 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:38.593894 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:38.594325 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:38.595729 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:38.597773 (+ 2044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:38.599475 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:38.600503 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:38.600676 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:38.604665 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:38.604679 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:38.604698 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:38.604754 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:38.608591 (+ 3837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:38.609431 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:38.609440 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:38.609449 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:38.609677 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:38.610069 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:38.610074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:38.610077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:38.610816 (+ 739us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:38.611132 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:38.611520 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 19:45:38.611984 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":346,"threads_started":1}
I20260807 19:45:38.617717 16299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } }
I20260807 19:45:38.618122 16299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:38.618244 16299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13947c19c459464cb19d9bf29abdf38d, State: Initialized, Role: FOLLOWER
I20260807 19:45:38.618844 16299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } }
I20260807 19:45:38.619056 16299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:45:38.619153 16299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:45:38.619271 16299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:38.621392 16299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } }
I20260807 19:45:38.621852 16299 leader_election.cc:302] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13947c19c459464cb19d9bf29abdf38d; no voters:
I20260807 19:45:38.622385 16299 leader_election.cc:288] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:45:38.622527 16305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:45:38.623198 16305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 1 LEADER]: Becoming Leader. State: Replica: 13947c19c459464cb19d9bf29abdf38d, State: Running, Role: LEADER
I20260807 19:45:38.623809 16305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } }
I20260807 19:45:38.624449 16299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:45:38.630231 16306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13947c19c459464cb19d9bf29abdf38d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } } }
I20260807 19:45:38.630645 16306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:38.634686 16307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13947c19c459464cb19d9bf29abdf38d. Latest consensus state: current_term: 1 leader_uuid: "13947c19c459464cb19d9bf29abdf38d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } } }
I20260807 19:45:38.635025 16307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: This master's current role is: LEADER
I20260807 19:45:38.635660 16313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:45:38.640349 16313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:45:38.648707 16313 catalog_manager.cc:1347] Generated new cluster ID: 3f960a5603ac46f9bc50465b12d6b1cf
I20260807 19:45:38.648833 16313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:45:38.672071 16313 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:45:38.673638 16313 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:45:38.683691 16313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Generated new TSK 0
I20260807 19:45:38.684892 16313 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:45:38.760372 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:37421
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:39.102355 16330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:39.102641 16330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:39.102785 16330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:39.110078 16330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:39.110206 16330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:39.110357 16330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:45:39.120250 16330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:37421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:39.121557 16330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:39.122869 16330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:39.135957 16336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:39.135986 16337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:39.136045 16339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:39.136665 16330 server_base.cc:1034] running on GCE node
I20260807 19:45:39.137295 16330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:39.138200 16330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:39.139413 16330 hybrid_clock.cc:648] HybridClock initialized: now 1786131939139362 us; error 37 us; skew 500 ppm
I20260807 19:45:39.139717 16330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:39 dist-test-slave-vq2q krb5kdc[16205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131939, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:39.142729 16330 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:45:39.144105 16330 webserver.cc:460] Webserver started at http://127.30.208.1:44953/ using document root <none> and password file <none>
I20260807 19:45:39.144768 16330 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:39.144882 16330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:39.145201 16330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:39.148200 16330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3"
format_stamp: "Formatted at 2026-08-07 19:45:39 on dist-test-slave-vq2q"
server_key: "386b718c59f96658447e588648199cb2"
server_key_iv: "82154591a58503a697620253f1c91702"
server_key_version: "encryptionkey@0"
I20260807 19:45:39.148947 16330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3"
format_stamp: "Formatted at 2026-08-07 19:45:39 on dist-test-slave-vq2q"
server_key: "386b718c59f96658447e588648199cb2"
server_key_iv: "82154591a58503a697620253f1c91702"
server_key_version: "encryptionkey@0"
I20260807 19:45:39.154234 16330 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:45:39.157848 16347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:39.159178 16330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:45:39.159338 16330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3"
format_stamp: "Formatted at 2026-08-07 19:45:39 on dist-test-slave-vq2q"
server_key: "386b718c59f96658447e588648199cb2"
server_key_iv: "82154591a58503a697620253f1c91702"
server_key_version: "encryptionkey@0"
I20260807 19:45:39.159533 16330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:39.181305 16330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:39.185734 16330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:39.186009 16330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:39.186956 16330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:39.188284 16330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:39.188359 16330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:39.188473 16330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:39.188521 16330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:39.219244 16330 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:33293
I20260807 19:45:39.219316 16470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:33293 every 8 connection(s)
I20260807 19:45:39.220521 16330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:45:39.221160 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 16330
I20260807 19:45:39.221434 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:45:39.222257 31552 external_mini_cluster.cc:1428] Setting key 12415ba673d34c726e5472ac6233b698
Aug 07 19:45:39 dist-test-slave-vq2q krb5kdc[16205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131939, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:39.241454 16473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:39.223249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:50987 (local address 127.30.208.62:37421)
0807 19:45:39.223756 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:39.223768 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:39.225102 (+ 1334us) server_negotiation.cc:407] Connection header received
0807 19:45:39.226801 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:39.226807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:39.226893 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:39.226988 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:39.228533 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:39.229330 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:39.230990 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:39.231415 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:39.231543 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:39.235334 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:39.235340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:39.235343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:39.235378 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:39.238369 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:39.239107 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:39.239111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:39.239113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:39.239238 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:39.239655 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:39.239659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:39.239661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:39.239944 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:39.240078 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:39.240829 (+ 751us) server_negotiation.cc:299] Negotiation successful
0807 19:45:39.241070 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":249,"threads_started":1}
I20260807 19:45:39.242481 16474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:39.223828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:50987)
0807 19:45:39.224697 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:39.224752 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:39.226529 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:39.227240 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:39.227248 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:39.227777 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:39.228354 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:39.228377 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:39.229497 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:39.229501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:39.230844 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:39.230857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:39.231652 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:39.231659 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:39.231888 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:39.232563 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:39.232586 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:39.235018 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:39.238516 (+ 3498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:39.238524 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:39.238538 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:39.238984 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:39.239350 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:39.239354 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:39.239356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:39.239545 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:39.240069 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:39.240078 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:39.240432 (+ 354us) client_negotiation.cc:769] Sending connection context
0807 19:45:39.240688 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 19:45:39.241283 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":343,"threads_started":1}
I20260807 19:45:39.244194 16471 heartbeater.cc:347] Connected to a master server at 127.30.208.62:37421
I20260807 19:45:39.244565 16471 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:39.245491 16471 heartbeater.cc:511] Master 127.30.208.62:37421 requested a full tablet report, sending...
I20260807 19:45:39.248066 16258 ts_manager.cc:194] Registered new tserver with Master: 6a9fb75e5e0340da8af60d54d2c3d4c3 (127.30.208.1:33293)
I20260807 19:45:39.250206 16258 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:50987
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:45:39.286020 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:37421
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:39.618233 16477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:39.618523 16477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:39.618654 16477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:39.626118 16477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:39.626245 16477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:39.626399 16477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:45:39.637583 16477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:37421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:39.639117 16477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:39.640933 16477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:39.655622 16485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:39.655716 16486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:39.655864 16488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:39.656956 16477 server_base.cc:1034] running on GCE node
I20260807 19:45:39.657680 16477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:39.658697 16477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:39.659940 16477 hybrid_clock.cc:648] HybridClock initialized: now 1786131939659874 us; error 58 us; skew 500 ppm
I20260807 19:45:39.660316 16477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:39 dist-test-slave-vq2q krb5kdc[16205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131939, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:39.663857 16477 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:45:39.665733 16477 webserver.cc:460] Webserver started at http://127.30.208.2:45767/ using document root <none> and password file <none>
I20260807 19:45:39.666476 16477 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:39.666590 16477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:39.666870 16477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:39.669986 16477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "63a96326893847a5a6f3c63e031a9676"
format_stamp: "Formatted at 2026-08-07 19:45:39 on dist-test-slave-vq2q"
server_key: "e40ccb92e33fdef1e76ececb236d9fce"
server_key_iv: "7e33bcff8596ed96dd971598214832fd"
server_key_version: "encryptionkey@0"
I20260807 19:45:39.670756 16477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "63a96326893847a5a6f3c63e031a9676"
format_stamp: "Formatted at 2026-08-07 19:45:39 on dist-test-slave-vq2q"
server_key: "e40ccb92e33fdef1e76ececb236d9fce"
server_key_iv: "7e33bcff8596ed96dd971598214832fd"
server_key_version: "encryptionkey@0"
I20260807 19:45:39.676599 16477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 19:45:39.681089 16495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:39.683805 16477 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260807 19:45:39.684188 16477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "63a96326893847a5a6f3c63e031a9676"
format_stamp: "Formatted at 2026-08-07 19:45:39 on dist-test-slave-vq2q"
server_key: "e40ccb92e33fdef1e76ececb236d9fce"
server_key_iv: "7e33bcff8596ed96dd971598214832fd"
server_key_version: "encryptionkey@0"
I20260807 19:45:39.684477 16477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:39.707738 16477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:39.712566 16477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:39.712858 16477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:39.714026 16477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:39.715417 16477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:39.715502 16477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:39.715626 16477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:39.715678 16477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:39.748733 16477 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:34111
I20260807 19:45:39.748804 16618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:34111 every 8 connection(s)
I20260807 19:45:39.750247 16477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:45:39.754045 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 16477
I20260807 19:45:39.754312 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:45:39.754900 31552 external_mini_cluster.cc:1428] Setting key ce26e1b8c915f4dbcd44e4e10947b5e4
Aug 07 19:45:39 dist-test-slave-vq2q krb5kdc[16205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131939, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:39.772658 16622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:39.753021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:55275 (local address 127.30.208.62:37421)
0807 19:45:39.753916 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:39.753929 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:39.754881 (+ 952us) server_negotiation.cc:407] Connection header received
0807 19:45:39.757717 (+ 2836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:39.757723 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:39.757808 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:39.757902 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:39.759574 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:39.760307 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:39.761967 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:39.762391 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:39.762505 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:39.766193 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:39.766200 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:39.766205 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:39.766248 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:39.769155 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:39.770185 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:39.770189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:39.770191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:39.770362 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:39.770874 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:39.770877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:39.770879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:39.771152 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:39.771261 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:39.772181 (+ 920us) server_negotiation.cc:299] Negotiation successful
0807 19:45:39.772377 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":550,"threads_started":1}
I20260807 19:45:39.774104 16623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:39.753702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:55275)
0807 19:45:39.754598 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:39.754649 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:39.757339 (+ 2690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:39.758180 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:39.758190 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:39.758778 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:39.759390 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:39.759410 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:39.760462 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:39.760466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:39.761818 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:39.761832 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:39.762625 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:39.762633 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:39.762856 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:39.763574 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:39.763599 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:39.765908 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:39.769367 (+ 3459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:39.769378 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:39.769400 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:39.770064 (+ 664us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:39.770473 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:39.770478 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:39.770482 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:39.770778 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:39.771271 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:39.771279 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:39.771728 (+ 449us) client_negotiation.cc:769] Sending connection context
0807 19:45:39.772074 (+ 346us) client_negotiation.cc:240] Negotiation successful
0807 19:45:39.772629 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":319,"threads_started":1}
I20260807 19:45:39.775882 16619 heartbeater.cc:347] Connected to a master server at 127.30.208.62:37421
I20260807 19:45:39.776474 16619 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:39.777967 16619 heartbeater.cc:511] Master 127.30.208.62:37421 requested a full tablet report, sending...
I20260807 19:45:39.780632 16258 ts_manager.cc:194] Registered new tserver with Master: 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111)
I20260807 19:45:39.781805 16258 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:55275
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:45:39.825505 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:37421
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:45:40.163401 16626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:45:40.163700 16626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:45:40.163844 16626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:45:40.171185 16626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:45:40.171311 16626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:45:40.171460 16626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:45:40.181387 16626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:37421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:45:40.182756 16626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:45:40.184096 16626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:45:40.197223 16636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:40.197294 16633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:45:40.197268 16634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:45:40.198282 16626 server_base.cc:1034] running on GCE node
I20260807 19:45:40.198886 16626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:45:40.199818 16626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:45:40.201025 16626 hybrid_clock.cc:648] HybridClock initialized: now 1786131940200975 us; error 44 us; skew 500 ppm
I20260807 19:45:40.201360 16626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:45:40 dist-test-slave-vq2q krb5kdc[16205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131940, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:45:40.204339 16626 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:45:40.205780 16626 webserver.cc:460] Webserver started at http://127.30.208.3:38727/ using document root <none> and password file <none>
I20260807 19:45:40.206427 16626 fs_manager.cc:359] Metadata directory not provided
I20260807 19:45:40.206543 16626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:45:40.206838 16626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:45:40.209709 16626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "a1d13b1d1b474bdaa50f36bc26b7c094"
format_stamp: "Formatted at 2026-08-07 19:45:40 on dist-test-slave-vq2q"
server_key: "c76679038d74e7d7321ca77ba429b52f"
server_key_iv: "17fe3f427c21ac757dd322255e00e7c5"
server_key_version: "encryptionkey@0"
I20260807 19:45:40.210438 16626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "a1d13b1d1b474bdaa50f36bc26b7c094"
format_stamp: "Formatted at 2026-08-07 19:45:40 on dist-test-slave-vq2q"
server_key: "c76679038d74e7d7321ca77ba429b52f"
server_key_iv: "17fe3f427c21ac757dd322255e00e7c5"
server_key_version: "encryptionkey@0"
I20260807 19:45:40.215444 16626 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:45:40.218930 16644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:40.220371 16626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:45:40.220533 16626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "a1d13b1d1b474bdaa50f36bc26b7c094"
format_stamp: "Formatted at 2026-08-07 19:45:40 on dist-test-slave-vq2q"
server_key: "c76679038d74e7d7321ca77ba429b52f"
server_key_iv: "17fe3f427c21ac757dd322255e00e7c5"
server_key_version: "encryptionkey@0"
I20260807 19:45:40.220718 16626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:45:40.244834 16626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:45:40.249074 16626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:45:40.249384 16626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:45:40.250399 16626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:45:40.251859 16626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:45:40.251933 16626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:40.252024 16626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:45:40.252066 16626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:45:40.253574 16471 heartbeater.cc:503] Master 127.30.208.62:37421 was elected leader, sending a full tablet report...
I20260807 19:45:40.283443 16626 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:43785
I20260807 19:45:40.283527 16769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:43785 every 8 connection(s)
I20260807 19:45:40.284698 16626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:45:40.294499 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 16626
I20260807 19:45:40.294670 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:45:40.295156 31552 external_mini_cluster.cc:1428] Setting key ed4c5329a75ecdfd18368d518e039f05
Aug 07 19:45:40 dist-test-slave-vq2q krb5kdc[16205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131940, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:45:40.303787 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.287091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:48563 (local address 127.30.208.62:37421)
0807 19:45:40.287487 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:40.287491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:40.288425 (+ 934us) server_negotiation.cc:407] Connection header received
0807 19:45:40.290338 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:40.290344 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:40.290437 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:40.290533 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:40.292007 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.292775 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.294401 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.294895 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.294998 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.298643 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:40.298648 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:40.298652 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:40.298691 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:40.301256 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:40.301892 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:40.301896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:40.301898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:40.302020 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:40.302345 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:40.302348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:40.302350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:40.302603 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:40.302732 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:40.303347 (+ 615us) server_negotiation.cc:299] Negotiation successful
0807 19:45:40.303512 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":200,"threads_started":1}
I20260807 19:45:40.304659 16773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.287528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:48563)
0807 19:45:40.288234 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:40.288323 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:40.290082 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:40.290766 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:40.290775 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:40.291294 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:45:40.291837 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:40.291855 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.292924 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.292928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:40.294231 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:40.294248 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.295118 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.295126 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:40.295421 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.296158 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:45:40.296187 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:45:40.298457 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:45:40.301367 (+ 2910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:40.301373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:40.301385 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:40.301794 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:40.302094 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:45:40.302097 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:45:40.302099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:45:40.302265 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:45:40.302714 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:45:40.302720 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:45:40.303009 (+ 289us) client_negotiation.cc:769] Sending connection context
0807 19:45:40.303221 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 19:45:40.303587 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":258,"threads_started":1}
I20260807 19:45:40.305979 16770 heartbeater.cc:347] Connected to a master server at 127.30.208.62:37421
I20260807 19:45:40.306281 16770 heartbeater.cc:464] Registering TS with master...
I20260807 19:45:40.306951 16770 heartbeater.cc:511] Master 127.30.208.62:37421 requested a full tablet report, sending...
I20260807 19:45:40.308276 16258 ts_manager.cc:194] Registered new tserver with Master: a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785)
I20260807 19:45:40.309342 16258 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:48563
I20260807 19:45:40.310530 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:45:40.328361 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.315519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34036 (local address 127.30.208.62:37421)
0807 19:45:40.315678 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:40.315684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:40.316063 (+ 379us) server_negotiation.cc:407] Connection header received
0807 19:45:40.316222 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:40.316227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:40.316301 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:40.316385 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:40.317831 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.318536 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.319887 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.320285 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.320393 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.322788 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:40.322793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:40.322796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:40.322827 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:40.325510 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:40.326173 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:40.326179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:40.326183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:40.326321 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:40.326720 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:40.326725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:40.326729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:40.327079 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:40.327319 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:40.327751 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 19:45:40.328019 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 19:45:40.332616 16258 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:45:40.339485 16258 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:45:40.364126 16786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.351633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48772 (local address 127.30.208.1:33293)
0807 19:45:40.354640 (+ 3007us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:40.354648 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:40.354678 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:45:40.354819 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:40.354829 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:40.355150 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:40.356904 (+ 1754us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:40.357147 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.358848 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.361614 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.362967 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.363110 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.363357 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:40.363616 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 19:45:40.363787 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2843,"thread_start_us":2671,"threads_started":1}
I20260807 19:45:40.364779 16787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.351683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:34111 (local address 127.0.0.1:41198)
0807 19:45:40.354620 (+ 2937us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:40.354634 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:40.354759 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:40.356159 (+ 1400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:40.356165 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:40.356192 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:40.356888 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:40.356904 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.359025 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.359031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:40.361938 (+ 2907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:40.361950 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.363902 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.363909 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:40.364097 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.364134 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:45:40.364234 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 19:45:40.364371 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2840,"mutex_wait_us":19,"thread_start_us":2695,"threads_started":1}
I20260807 19:45:40.365371 16785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.351473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:33293 (local address 127.0.0.1:48772)
0807 19:45:40.353348 (+ 1875us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:40.353404 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:40.353580 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:40.355440 (+ 1860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:40.355449 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:40.355484 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:40.356666 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:40.356682 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.359026 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.359033 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:40.361472 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:40.361485 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.363176 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.363184 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:40.363373 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.363421 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 19:45:40.363546 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 19:45:40.364927 (+ 1381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"mutex_wait_us":39,"thread_start_us":621,"threads_started":1}
I20260807 19:45:40.365638 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.351941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41198 (local address 127.30.208.2:34111)
0807 19:45:40.354198 (+ 2257us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:40.354209 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:40.354875 (+ 666us) server_negotiation.cc:407] Connection header received
0807 19:45:40.354977 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:40.354985 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:40.355707 (+ 722us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:40.355895 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:40.357129 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.358836 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.362113 (+ 3277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.363469 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.363583 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.363768 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:40.365203 (+ 1435us) server_negotiation.cc:299] Negotiation successful
0807 19:45:40.365345 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2085,"thread_start_us":1962,"threads_started":1}
I20260807 19:45:40.368804 16789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.354782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:43785 (local address 127.0.0.1:41786)
0807 19:45:40.359526 (+ 4744us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:40.359547 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:40.359657 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:40.361901 (+ 2244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:40.361906 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:40.361935 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:40.362261 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:40.362270 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.363956 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.363961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:40.366267 (+ 2306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:40.366284 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.368054 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.368061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:40.368224 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.368256 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:45:40.368351 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 19:45:40.368476 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4629,"thread_start_us":4482,"threads_started":1}
I20260807 19:45:40.368976 16548 tablet_service.cc:1467] Processing CreateTablet for tablet 9effab721c004a98a5d6884d644ceada (DEFAULT_TABLE table=test-table [id=377f09b2c7944d8aacacdf27ea8d578a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:40.369077 16401 tablet_service.cc:1467] Processing CreateTablet for tablet 9effab721c004a98a5d6884d644ceada (DEFAULT_TABLE table=test-table [id=377f09b2c7944d8aacacdf27ea8d578a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:40.370218 16790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.355108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41786 (local address 127.30.208.3:43785)
0807 19:45:40.361042 (+ 5934us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:40.361050 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:40.361081 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:45:40.361167 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:40.361200 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:40.361434 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:40.361614 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:40.362420 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.363786 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.366452 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.367837 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.367973 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.368184 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:40.368422 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 19:45:40.368582 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5759,"thread_start_us":5590,"threads_started":1}
I20260807 19:45:40.370263 16548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9effab721c004a98a5d6884d644ceada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:40.370517 16401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9effab721c004a98a5d6884d644ceada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:40.377110 16701 tablet_service.cc:1467] Processing CreateTablet for tablet 9effab721c004a98a5d6884d644ceada (DEFAULT_TABLE table=test-table [id=377f09b2c7944d8aacacdf27ea8d578a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:45:40.378607 16701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9effab721c004a98a5d6884d644ceada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:45:40.381592 16791 tablet_bootstrap.cc:492] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676: Bootstrap starting.
I20260807 19:45:40.384058 16793 tablet_bootstrap.cc:492] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3: Bootstrap starting.
I20260807 19:45:40.385926 16791 tablet_bootstrap.cc:654] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:40.387233 16791 log.cc:824] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:40.387634 16793 tablet_bootstrap.cc:654] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:40.388972 16793 log.cc:824] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:40.392345 16791 tablet_bootstrap.cc:492] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676: No bootstrap required, opened a new log
I20260807 19:45:40.392357 16793 tablet_bootstrap.cc:492] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3: No bootstrap required, opened a new log
I20260807 19:45:40.392630 16793 ts_tablet_manager.cc:1397] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260807 19:45:40.392699 16791 ts_tablet_manager.cc:1397] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260807 19:45:40.393038 16796 tablet_bootstrap.cc:492] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094: Bootstrap starting.
I20260807 19:45:40.396534 16796 tablet_bootstrap.cc:654] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094: Neither blocks nor log segments found. Creating new log.
I20260807 19:45:40.397832 16796 log.cc:824] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094: Log is configured to *not* fsync() on all Append() calls
I20260807 19:45:40.400063 16791 raft_consensus.cc:348] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:45:40.400409 16791 raft_consensus.cc:374] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:40.400499 16791 raft_consensus.cc:729] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a96326893847a5a6f3c63e031a9676, State: Initialized, Role: FOLLOWER
I20260807 19:45:40.400982 16796 tablet_bootstrap.cc:492] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094: No bootstrap required, opened a new log
I20260807 19:45:40.401064 16791 consensus_queue.cc:260] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:45:40.401286 16796 ts_tablet_manager.cc:1397] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260807 19:45:40.402196 16619 heartbeater.cc:503] Master 127.30.208.62:37421 was elected leader, sending a full tablet report...
I20260807 19:45:40.402436 16793 raft_consensus.cc:348] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:45:40.402853 16793 raft_consensus.cc:374] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:40.402896 16791 ts_tablet_manager.cc:1428] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676: Time spent starting tablet: real 0.010s user 0.002s sys 0.007s
I20260807 19:45:40.402948 16793 raft_consensus.cc:729] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a9fb75e5e0340da8af60d54d2c3d4c3, State: Initialized, Role: FOLLOWER
I20260807 19:45:40.403926 16793 consensus_queue.cc:260] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:45:40.405686 16793 ts_tablet_manager.cc:1428] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260807 19:45:40.411504 16796 raft_consensus.cc:348] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:45:40.411864 16796 raft_consensus.cc:374] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:45:40.411965 16796 raft_consensus.cc:729] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d13b1d1b474bdaa50f36bc26b7c094, State: Initialized, Role: FOLLOWER
I20260807 19:45:40.412551 16796 consensus_queue.cc:260] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:45:40.413666 16770 heartbeater.cc:503] Master 127.30.208.62:37421 was elected leader, sending a full tablet report...
I20260807 19:45:40.414327 16796 ts_tablet_manager.cc:1428] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260807 19:45:40.443925 16799 raft_consensus.cc:482] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:45:40.444247 16799 raft_consensus.cc:504] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:45:40.446077 16799 leader_election.cc:288] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111), a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785)
I20260807 19:45:40.452670 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.446543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:58239 (local address 127.30.208.2:34111)
0807 19:45:40.446750 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:40.446755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:40.447159 (+ 404us) server_negotiation.cc:407] Connection header received
0807 19:45:40.447385 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:40.447391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:40.447474 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:40.447580 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:40.448166 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.448922 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.450672 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.451676 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.451830 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.451900 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:40.452287 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 19:45:40.452417 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:45:40.452715 16802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.446401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:34111 (local address 127.30.208.1:58239)
0807 19:45:40.447031 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:40.447051 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:40.447218 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:40.447654 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:40.447659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:40.447688 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:40.448018 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:40.448027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.449051 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.449055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:40.450515 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:40.450528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.451900 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.451907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:40.452094 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.452126 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:45:40.452218 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 19:45:40.452348 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":313,"threads_started":1}
I20260807 19:45:40.453438 16790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.447354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:50967 (local address 127.30.208.3:43785)
0807 19:45:40.447553 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:40.447559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:40.447886 (+ 327us) server_negotiation.cc:407] Connection header received
0807 19:45:40.448059 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:40.448064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:40.448125 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:40.448197 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:45:40.448719 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.449500 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.451156 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.452131 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.452292 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.452360 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:40.453033 (+ 673us) server_negotiation.cc:299] Negotiation successful
0807 19:45:40.453163 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:45:40.453471 16803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.447178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:43785 (local address 127.30.208.1:50967)
0807 19:45:40.447769 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 19:45:40.447787 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:45:40.447909 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:45:40.448226 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:45:40.448230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:45:40.448254 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:45:40.448558 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:40.448568 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.449637 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.449641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:40.451008 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:45:40.451020 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.452533 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.452539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:45:40.452710 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.452741 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 19:45:40.452986 (+ 245us) client_negotiation.cc:240] Negotiation successful
0807 19:45:40.453109 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":291,"threads_started":1}
I20260807 19:45:40.453635 16569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9effab721c004a98a5d6884d644ceada" candidate_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a96326893847a5a6f3c63e031a9676" is_pre_election: true
I20260807 19:45:40.454175 16569 raft_consensus.cc:2454] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a9fb75e5e0340da8af60d54d2c3d4c3 in term 0.
I20260807 19:45:40.454361 16722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9effab721c004a98a5d6884d644ceada" candidate_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" is_pre_election: true
I20260807 19:45:40.454826 16349 leader_election.cc:302] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63a96326893847a5a6f3c63e031a9676, 6a9fb75e5e0340da8af60d54d2c3d4c3; no voters:
I20260807 19:45:40.454886 16722 raft_consensus.cc:2454] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a9fb75e5e0340da8af60d54d2c3d4c3 in term 0.
I20260807 19:45:40.455302 16799 raft_consensus.cc:2781] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:45:40.455473 16799 raft_consensus.cc:482] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:45:40.455579 16799 raft_consensus.cc:3037] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:40.457855 16799 raft_consensus.cc:504] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:45:40.458441 16799 leader_election.cc:288] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111), a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785)
I20260807 19:45:40.458928 16569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9effab721c004a98a5d6884d644ceada" candidate_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a96326893847a5a6f3c63e031a9676"
I20260807 19:45:40.459123 16569 raft_consensus.cc:3037] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:40.459311 16722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9effab721c004a98a5d6884d644ceada" candidate_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094"
I20260807 19:45:40.459518 16722 raft_consensus.cc:3037] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:45:40.462492 16569 raft_consensus.cc:2454] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a9fb75e5e0340da8af60d54d2c3d4c3 in term 1.
I20260807 19:45:40.462716 16722 raft_consensus.cc:2454] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a9fb75e5e0340da8af60d54d2c3d4c3 in term 1.
I20260807 19:45:40.463325 16349 leader_election.cc:302] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63a96326893847a5a6f3c63e031a9676, 6a9fb75e5e0340da8af60d54d2c3d4c3; no voters:
I20260807 19:45:40.463650 16799 raft_consensus.cc:2781] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:45:40.464455 16799 raft_consensus.cc:686] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 LEADER]: Becoming Leader. State: Replica: 6a9fb75e5e0340da8af60d54d2c3d4c3, State: Running, Role: LEADER
I20260807 19:45:40.465039 16799 consensus_queue.cc:237] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:45:40.470041 16257 catalog_manager.cc:5495] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a9fb75e5e0340da8af60d54d2c3d4c3 (127.30.208.1). New cstate: current_term: 1 leader_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } health_report { overall_health: UNKNOWN } } }
W20260807 19:45:40.473038 16472 tablet.cc:2369] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:45:40.489035 16786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.483074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.30.208.1:33293)
0807 19:45:40.483289 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:40.483295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:40.483370 (+ 75us) server_negotiation.cc:407] Connection header received
0807 19:45:40.483592 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:40.483597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:40.483686 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:40.483816 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:45:40.484191 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.484942 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.486944 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.487360 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.487461 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.487960 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:45:40.488486 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:45:40.488609 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:40.488702 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 19:45:40.488773 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:45:40.497506 16569 raft_consensus.cc:1261] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676 [term 1 FOLLOWER]: Refusing update from remote peer 6a9fb75e5e0340da8af60d54d2c3d4c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:45:40.497505 16722 raft_consensus.cc:1261] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094 [term 1 FOLLOWER]: Refusing update from remote peer 6a9fb75e5e0340da8af60d54d2c3d4c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:45:40.498623 16799 consensus_queue.cc:1035] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:45:40.498974 16804 consensus_queue.cc:1035] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:45:40.502123 16621 tablet.cc:2369] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:45:40.513676 16808 mvcc.cc:205] Tried to move back new op lower bound from 7315996428266393600 to 7315996428149121024. Current Snapshot: MvccSnapshot[applied={T|T < 7315996428266393600}]
I20260807 19:45:40.531148 16258 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:45:40.531486 16258 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:45:40.535436 16258 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Sending DeleteTablet for 3 replicas of tablet 9effab721c004a98a5d6884d644ceada
I20260807 19:45:40.536494 16401 tablet_service.cc:1514] Processing DeleteTablet for tablet 9effab721c004a98a5d6884d644ceada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:40 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:48772
I20260807 19:45:40.536689 16548 tablet_service.cc:1514] Processing DeleteTablet for tablet 9effab721c004a98a5d6884d644ceada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:40 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:41198
I20260807 19:45:40.536845 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 16330
I20260807 19:45:40.537475 16701 tablet_service.cc:1514] Processing DeleteTablet for tablet 9effab721c004a98a5d6884d644ceada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:45:40 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:41786
I20260807 19:45:40.537853 16819 tablet_replica.cc:330] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676: stopping tablet replica
I20260807 19:45:40.538223 16819 raft_consensus.cc:2229] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:45:40.538687 16819 raft_consensus.cc:2258] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:40.541690 16817 tablet_replica.cc:330] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3: stopping tablet replica
I20260807 19:45:40.542330 16820 tablet_replica.cc:330] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094: stopping tablet replica
I20260807 19:45:40.542757 16820 raft_consensus.cc:2229] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:45:40.543309 16820 raft_consensus.cc:2258] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:40.543463 16819 ts_tablet_manager.cc:1905] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:45:40.544380 16817 raft_consensus.cc:2229] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:45:40.545094 16817 raft_consensus.cc:2258] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:45:40.547039 16820 ts_tablet_manager.cc:1905] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:45:40.550532 16817 ts_tablet_manager.cc:1905] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:45:40.551779 16819 ts_tablet_manager.cc:1918] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:45:40.552007 16819 log.cc:1196] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/9effab721c004a98a5d6884d644ceada
I20260807 19:45:40.552629 16819 ts_tablet_manager.cc:1939] T 9effab721c004a98a5d6884d644ceada P 63a96326893847a5a6f3c63e031a9676: Deleting consensus metadata
I20260807 19:45:40.554677 16242 catalog_manager.cc:4841] TS 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111): tablet 9effab721c004a98a5d6884d644ceada (table test-table [id=377f09b2c7944d8aacacdf27ea8d578a]) successfully deleted
Aug 07 19:45:40 dist-test-slave-vq2q krb5kdc[16205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:45:40.557952 16817 ts_tablet_manager.cc:1918] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:45:40.558151 16817 log.cc:1196] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/9effab721c004a98a5d6884d644ceada
I20260807 19:45:40.558723 16817 ts_tablet_manager.cc:1939] T 9effab721c004a98a5d6884d644ceada P 6a9fb75e5e0340da8af60d54d2c3d4c3: Deleting consensus metadata
I20260807 19:45:40.560642 16820 ts_tablet_manager.cc:1918] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:45:40.560740 16244 catalog_manager.cc:4841] TS 6a9fb75e5e0340da8af60d54d2c3d4c3 (127.30.208.1:33293): tablet 9effab721c004a98a5d6884d644ceada (table test-table [id=377f09b2c7944d8aacacdf27ea8d578a]) successfully deleted
I20260807 19:45:40.560930 16820 log.cc:1196] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/9effab721c004a98a5d6884d644ceada
I20260807 19:45:40.561549 16820 ts_tablet_manager.cc:1939] T 9effab721c004a98a5d6884d644ceada P a1d13b1d1b474bdaa50f36bc26b7c094: Deleting consensus metadata
I20260807 19:45:40.563460 16242 catalog_manager.cc:4841] TS a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785): tablet 9effab721c004a98a5d6884d644ceada (table test-table [id=377f09b2c7944d8aacacdf27ea8d578a]) successfully deleted
I20260807 19:45:40.564354 16786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.538300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48796 (local address 127.30.208.1:33293)
0807 19:45:40.538474 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:40.538480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:40.544156 (+ 5676us) server_negotiation.cc:407] Connection header received
0807 19:45:40.544444 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:40.544450 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:40.544565 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:40.544643 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:40.547278 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.548363 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.550338 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.551058 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.551159 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.556760 (+ 5601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:40.556774 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:40.556792 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:40.556846 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:40.560977 (+ 4131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:40.561859 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:40.561872 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:40.561874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:40.562031 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:40.562406 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:40.562410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:40.562412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:40.562829 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:40.563109 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:40.563650 (+ 541us) server_negotiation.cc:299] Negotiation successful
0807 19:45:40.563956 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:45:40.565021 16466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:40.664474 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 16330
I20260807 19:45:40.684656 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 16477
Aug 07 19:45:40 dist-test-slave-vq2q krb5kdc[16205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:45:40.699887 16788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.685418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41200 (local address 127.30.208.2:34111)
0807 19:45:40.685573 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:40.685577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:40.685615 (+ 38us) server_negotiation.cc:407] Connection header received
0807 19:45:40.685781 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:40.685786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:40.685850 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:40.685955 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:40.687270 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.687994 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.689455 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.689936 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.690040 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.694300 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:40.694317 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:40.694334 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:40.694372 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:40.697356 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:40.698084 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:40.698091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:40.698093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:40.698250 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:40.698560 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:40.698563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:40.698565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:40.698983 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:40.699236 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:40.699449 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 19:45:40.699624 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:45:40.700541 16614 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:40.790603 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 16477
I20260807 19:45:40.808226 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 16626
Aug 07 19:45:40 dist-test-slave-vq2q krb5kdc[16205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:45:40.824747 16790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:45:40.809030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41800 (local address 127.30.208.3:43785)
0807 19:45:40.809239 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:45:40.809245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:45:40.809264 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:45:40.809459 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:45:40.809465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:45:40.809540 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:45:40.809643 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:45:40.810974 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.811749 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.813155 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:45:40.813597 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:45:40.813741 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:45:40.818112 (+ 4371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:45:40.818131 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:45:40.818147 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:45:40.818185 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:45:40.821807 (+ 3622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:40.822593 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:40.822601 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:40.822604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:40.822764 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:45:40.823099 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:45:40.823102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:45:40.823104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:45:40.823488 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:45:40.823712 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 19:45:40.824247 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 19:45:40.824451 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 19:45:40.825448 16765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:40.916721 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 16626
I20260807 19:45:40.934476 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 16222
I20260807 19:45:40.935706 16294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:45:41.046685 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 16222
I20260807 19:46:01.068687 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:37421
--webserver_interface=127.30.208.62
--webserver_port=40599
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:37421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:01.417587 16828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:01.417907 16828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:01.418043 16828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:01.425290 16828 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 19:46:01.425395 16828 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 19:46:01.425450 16828 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 19:46:01.425524 16828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:46:01.425598 16828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:01.425711 16828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:01.425770 16828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:46:01.425848 16828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:46:01.435715 16828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:37421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:37421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=40599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:01.437047 16828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:01.438426 16828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:01.461838 16839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:01.461906 16842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:01.461838 16837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:01.462623 16828 server_base.cc:1034] running on GCE node
I20260807 19:46:01.463508 16828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:01.465071 16828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:01.466315 16828 hybrid_clock.cc:648] HybridClock initialized: now 1786131961466246 us; error 49 us; skew 500 ppm
I20260807 19:46:01.466718 16828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:01 dist-test-slave-vq2q krb5kdc[16205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131961, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:01.470434 16828 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:46:01.472123 16828 webserver.cc:460] Webserver started at http://127.30.208.62:40599/ using document root <none> and password file <none>
I20260807 19:46:01.472891 16828 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:01.473009 16828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:01.479151 16828 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 19:46:01.482259 16849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:01.483803 16828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:46:01.483991 16828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "13947c19c459464cb19d9bf29abdf38d"
format_stamp: "Formatted at 2026-08-07 19:45:38 on dist-test-slave-vq2q"
server_key: "2cb9b4ea3254fe45dd34d9bd1e56471e"
server_key_iv: "854738b13ad05a167f7a5ca5bcbf0dcc"
server_key_version: "encryptionkey@0"
I20260807 19:46:01.485112 16828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:01.498407 16828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:01.502799 16828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:01.503077 16828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:01.521855 16828 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:37421
I20260807 19:46:01.521900 16905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:37421 every 8 connection(s)
I20260807 19:46:01.523378 16828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:46:01.530094 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 16828
I20260807 19:46:01.530814 16906 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:46:01.531031 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:33293
--local_ip_for_outbound_sockets=127.30.208.1
--tserver_master_addrs=127.30.208.62:37421
--webserver_port=44953
--webserver_interface=127.30.208.1
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:46:01.534483 16906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Bootstrap starting.
I20260807 19:46:01.555238 16906 log.cc:824] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:01.568149 16906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:46:01.568692 16906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Bootstrap complete.
I20260807 19:46:01.577973 16906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } }
I20260807 19:46:01.578936 16906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13947c19c459464cb19d9bf29abdf38d, State: Initialized, Role: FOLLOWER
I20260807 19:46:01.579558 16906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } }
I20260807 19:46:01.579802 16906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:46:01.579924 16906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:46:01.580084 16906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 1 FOLLOWER]: Advancing to term 2
I20260807 19:46:01.584120 16906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } }
I20260807 19:46:01.584626 16906 leader_election.cc:302] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13947c19c459464cb19d9bf29abdf38d; no voters:
I20260807 19:46:01.585326 16906 leader_election.cc:288] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 19:46:01.585422 16910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 2 FOLLOWER]: Leader election won for term 2
I20260807 19:46:01.587715 16906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:46:01.592538 16910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 2 LEADER]: Becoming Leader. State: Replica: 13947c19c459464cb19d9bf29abdf38d, State: Running, Role: LEADER
I20260807 19:46:01.593111 16910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } }
I20260807 19:46:01.595709 16911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "13947c19c459464cb19d9bf29abdf38d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } } }
I20260807 19:46:01.596140 16911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:01.596678 16912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13947c19c459464cb19d9bf29abdf38d. Latest consensus state: current_term: 2 leader_uuid: "13947c19c459464cb19d9bf29abdf38d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } } }
I20260807 19:46:01.596951 16912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:01.603710 16921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:46:01.611644 16921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:46:01.613363 16921 catalog_manager.cc:1259] Loaded cluster ID: 3f960a5603ac46f9bc50465b12d6b1cf
I20260807 19:46:01.613487 16921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:46:01.618046 16921 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:46:01.619841 16921 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Loaded TSK: 0
I20260807 19:46:01.645905 16921 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Generated new TSK 1
I20260807 19:46:01.646309 16921 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 19:46:01.915335 16908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:01.915648 16908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:01.915787 16908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:01.923243 16908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:01.923389 16908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:01.923548 16908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:46:01.933816 16908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:33293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=44953
--tserver_master_addrs=127.30.208.62:37421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:01.935159 16908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:01.936518 16908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:01.951848 16939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:01.951850 16937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:01.953209 16908 server_base.cc:1034] running on GCE node
W20260807 19:46:01.953495 16936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:01.954218 16908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:01.955260 16908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:01.956489 16908 hybrid_clock.cc:648] HybridClock initialized: now 1786131961956427 us; error 54 us; skew 500 ppm
I20260807 19:46:01.956852 16908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:01 dist-test-slave-vq2q krb5kdc[16205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131961, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:01.960443 16908 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:46:01.962409 16908 webserver.cc:460] Webserver started at http://127.30.208.1:44953/ using document root <none> and password file <none>
I20260807 19:46:01.963132 16908 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:01.963289 16908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:01.968874 16908 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:46:01.971727 16946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:01.973094 16908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:46:01.973281 16908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3"
format_stamp: "Formatted at 2026-08-07 19:45:39 on dist-test-slave-vq2q"
server_key: "386b718c59f96658447e588648199cb2"
server_key_iv: "82154591a58503a697620253f1c91702"
server_key_version: "encryptionkey@0"
I20260807 19:46:01.974267 16908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:02.001021 16908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:02.005482 16908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:02.005756 16908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:02.006922 16908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:02.008334 16908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:02.008441 16908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:02.008555 16908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:02.008632 16908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:02.043922 16908 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:33293
I20260807 19:46:02.043980 17074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:33293 every 8 connection(s)
I20260807 19:46:02.045802 16908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:46:02.051486 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 16908
I20260807 19:46:02.052321 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:34111
--local_ip_for_outbound_sockets=127.30.208.2
--tserver_master_addrs=127.30.208.62:37421
--webserver_port=45767
--webserver_interface=127.30.208.2
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:46:02 dist-test-slave-vq2q krb5kdc[16205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131961, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:02.073891 17078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:02.049548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:43901)
0807 19:46:02.050394 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:02.050447 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:02.052254 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:02.054038 (+ 1784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:02.054049 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:02.054781 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:02.055485 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:02.055509 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.058034 (+ 2525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.058041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:02.059897 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:02.059913 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.060959 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.060967 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:02.061265 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:02.061901 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:02.061924 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:02.064147 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:02.068813 (+ 4666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:02.068822 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:02.068837 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:02.069435 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:02.069976 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:02.069981 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:02.069984 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:02.070225 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:02.071228 (+ 1003us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:02.071235 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:02.071619 (+ 384us) client_negotiation.cc:769] Sending connection context
0807 19:46:02.071969 (+ 350us) client_negotiation.cc:240] Negotiation successful
0807 19:46:02.072493 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":319,"threads_started":1}
I20260807 19:46:02.073975 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:02.049167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:43901 (local address 127.30.208.62:37421)
0807 19:46:02.049859 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:02.049871 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:02.052593 (+ 2722us) server_negotiation.cc:407] Connection header received
0807 19:46:02.052925 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:02.052943 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:02.053446 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:02.053864 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:02.055723 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.057851 (+ 2128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.060054 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.060704 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.060898 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:02.064484 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:02.064496 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:02.064516 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:02.064573 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:02.068563 (+ 3990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:02.069585 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:02.069596 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:02.069603 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:02.069849 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:02.070376 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:02.070382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:02.070386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:02.071060 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:02.071269 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:02.072266 (+ 997us) server_negotiation.cc:299] Negotiation successful
0807 19:46:02.072720 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":257,"threads_started":1}
I20260807 19:46:02.077528 17075 heartbeater.cc:347] Connected to a master server at 127.30.208.62:37421
I20260807 19:46:02.077852 17075 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:02.078735 17075 heartbeater.cc:511] Master 127.30.208.62:37421 requested a full tablet report, sending...
I20260807 19:46:02.081503 16867 ts_manager.cc:194] Registered new tserver with Master: 6a9fb75e5e0340da8af60d54d2c3d4c3 (127.30.208.1:33293)
I20260807 19:46:02.083570 16867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:43901
W20260807 19:46:02.400238 17079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:02.400530 17079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:02.400629 17079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:02.407972 17079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:02.408089 17079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:02.408222 17079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:46:02.418128 17079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:34111
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=45767
--tserver_master_addrs=127.30.208.62:37421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:02.419396 17079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:02.420650 17079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:02.433281 17084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:02.433416 17085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:02.433305 17087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:02.434468 17079 server_base.cc:1034] running on GCE node
I20260807 19:46:02.435055 17079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:02.435977 17079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:02.437218 17079 hybrid_clock.cc:648] HybridClock initialized: now 1786131962437140 us; error 37 us; skew 500 ppm
I20260807 19:46:02.437527 17079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:02 dist-test-slave-vq2q krb5kdc[16205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131962, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:02.440659 17079 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:46:02.442179 17079 webserver.cc:460] Webserver started at http://127.30.208.2:45767/ using document root <none> and password file <none>
I20260807 19:46:02.442852 17079 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:02.442983 17079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:02.448292 17079 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 19:46:02.451033 17095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:02.452364 17079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:46:02.452531 17079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "63a96326893847a5a6f3c63e031a9676"
format_stamp: "Formatted at 2026-08-07 19:45:39 on dist-test-slave-vq2q"
server_key: "e40ccb92e33fdef1e76ececb236d9fce"
server_key_iv: "7e33bcff8596ed96dd971598214832fd"
server_key_version: "encryptionkey@0"
I20260807 19:46:02.453821 17079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:02.466501 17079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:02.470690 17079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:02.470919 17079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:02.471897 17079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:02.473286 17079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:02.473387 17079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:02.473489 17079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:02.473558 17079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:02.502839 17079 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:34111
I20260807 19:46:02.502904 17226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:34111 every 8 connection(s)
I20260807 19:46:02.504207 17079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:46:02.506544 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 17079
I20260807 19:46:02.507246 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:43785
--local_ip_for_outbound_sockets=127.30.208.3
--tserver_master_addrs=127.30.208.62:37421
--webserver_port=38727
--webserver_interface=127.30.208.3
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 19:46:02 dist-test-slave-vq2q krb5kdc[16205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131962, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:02.526423 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:02.506976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:36707 (local address 127.30.208.62:37421)
0807 19:46:02.507192 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:02.507198 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:02.508505 (+ 1307us) server_negotiation.cc:407] Connection header received
0807 19:46:02.510400 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:02.510406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:02.510487 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:02.510600 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:02.512063 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.512999 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.514904 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.515505 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.515627 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:02.519631 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:02.519637 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:02.519641 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:02.519685 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:02.522855 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:02.523791 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:02.523796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:02.523799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:02.523966 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:02.524442 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:02.524446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:02.524449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:02.524827 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:02.524970 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:02.525878 (+ 908us) server_negotiation.cc:299] Negotiation successful
0807 19:46:02.526102 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:46:02.527578 17229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:02.507485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:36707)
0807 19:46:02.508322 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:02.508368 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:02.510130 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:02.510819 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:02.510827 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:02.511334 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:02.511888 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:02.511906 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.513155 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.513160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:02.514746 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:02.514768 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.515706 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.515711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:02.515954 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:02.516639 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:02.516665 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:02.519373 (+ 2708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:02.523011 (+ 3638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:02.523020 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:02.523036 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:02.523629 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:02.524069 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:02.524073 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:02.524077 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:02.524315 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:02.524981 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:02.524990 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:02.525427 (+ 437us) client_negotiation.cc:769] Sending connection context
0807 19:46:02.525750 (+ 323us) client_negotiation.cc:240] Negotiation successful
0807 19:46:02.526265 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":326,"threads_started":1}
I20260807 19:46:02.529379 17227 heartbeater.cc:347] Connected to a master server at 127.30.208.62:37421
I20260807 19:46:02.529716 17227 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:02.530555 17227 heartbeater.cc:511] Master 127.30.208.62:37421 requested a full tablet report, sending...
I20260807 19:46:02.532336 16867 ts_manager.cc:194] Registered new tserver with Master: 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111)
I20260807 19:46:02.533658 16867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:36707
W20260807 19:46:02.847700 17230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:02.847990 17230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:02.848119 17230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:02.855412 17230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:02.855528 17230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:02.855677 17230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:46:02.865509 17230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:43785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=38727
--tserver_master_addrs=127.30.208.62:37421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:02.866830 17230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:02.868152 17230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:02.881145 17237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:02.881546 17239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:02.881085 17236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:02.881713 17230 server_base.cc:1034] running on GCE node
I20260807 19:46:02.882356 17230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:02.883285 17230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:02.884485 17230 hybrid_clock.cc:648] HybridClock initialized: now 1786131962884435 us; error 32 us; skew 500 ppm
I20260807 19:46:02.884797 17230 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:02 dist-test-slave-vq2q krb5kdc[16205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131962, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:02.887984 17230 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:46:02.889590 17230 webserver.cc:460] Webserver started at http://127.30.208.3:38727/ using document root <none> and password file <none>
I20260807 19:46:02.890292 17230 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:02.890422 17230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:02.896678 17230 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 19:46:02.899619 17248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:02.900970 17230 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:46:02.901127 17230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "a1d13b1d1b474bdaa50f36bc26b7c094"
format_stamp: "Formatted at 2026-08-07 19:45:40 on dist-test-slave-vq2q"
server_key: "c76679038d74e7d7321ca77ba429b52f"
server_key_iv: "17fe3f427c21ac757dd322255e00e7c5"
server_key_version: "encryptionkey@0"
I20260807 19:46:02.902127 17230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:02.920463 17230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:02.924645 17230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:02.924909 17230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:02.925995 17230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:02.927390 17230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:02.927481 17230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:02.927587 17230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:02.927656 17230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:02.959038 17230 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:43785
I20260807 19:46:02.959089 17372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:43785 every 8 connection(s)
I20260807 19:46:02.960434 17230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:46:02.969447 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 17230
Aug 07 19:46:02 dist-test-slave-vq2q krb5kdc[16205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131962, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:02.982139 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:02.962953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:35611 (local address 127.30.208.62:37421)
0807 19:46:02.963169 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:02.963175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:02.964295 (+ 1120us) server_negotiation.cc:407] Connection header received
0807 19:46:02.966166 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:02.966171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:02.966241 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:02.966330 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:02.967858 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.968626 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.970336 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.971053 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.971191 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:02.975370 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:02.975374 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:02.975378 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:02.975411 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:02.978563 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:02.979520 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:02.979524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:02.979527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:02.979672 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:02.980124 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:02.980127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:02.980129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:02.980430 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:02.980581 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:02.981497 (+ 916us) server_negotiation.cc:299] Negotiation successful
0807 19:46:02.981762 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 19:46:02.983076 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:02.963398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:35611)
0807 19:46:02.964126 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:02.964169 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:02.965911 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:02.966536 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:02.966544 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:02.967111 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:02.967666 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:02.967694 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.968863 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.968867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:02.970200 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:02.970212 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.971301 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.971310 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:02.971562 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:02.972312 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:02.972343 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:02.975134 (+ 2791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:02.978710 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:02.978719 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:02.978737 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:02.979366 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:02.979773 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:02.979778 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:02.979781 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:02.980027 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:02.980550 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:02.980556 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:02.980882 (+ 326us) client_negotiation.cc:769] Sending connection context
0807 19:46:02.981167 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 19:46:02.981737 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":291,"threads_started":1}
I20260807 19:46:02.984586 17373 heartbeater.cc:347] Connected to a master server at 127.30.208.62:37421
I20260807 19:46:02.984877 17373 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:02.985641 17373 heartbeater.cc:511] Master 127.30.208.62:37421 requested a full tablet report, sending...
I20260807 19:46:02.986697 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:02.970463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35830 (local address 127.30.208.62:37421)
0807 19:46:02.971276 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:02.971281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:02.971297 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:46:02.971342 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:02.971347 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:02.971402 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:02.971488 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:02.973760 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.974764 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.976249 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.976957 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.977091 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:02.980430 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:02.980436 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:02.980440 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:02.980471 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:02.984028 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:02.984679 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:02.984685 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:02.984688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:02.984834 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:02.985238 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:02.985243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:02.985247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:02.985503 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:02.985766 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:02.986113 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 19:46:02.986336 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":565,"threads_started":1}
I20260807 19:46:02.987133 16867 ts_manager.cc:194] Registered new tserver with Master: a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785)
I20260807 19:46:02.987963 16867 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:35611
I20260807 19:46:02.990669 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:46:03.001754 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:02.996074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35836 (local address 127.30.208.62:37421)
0807 19:46:02.996292 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:02.996299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:02.996535 (+ 236us) server_negotiation.cc:407] Connection header received
0807 19:46:02.996792 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:02.996796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:02.996928 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:02.997063 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:46:02.997541 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:02.998257 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:02.999712 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.000126 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.000233 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.000775 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:46:03.001058 (+ 283us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 19:46:03.001470 (+ 412us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35836: token expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 19:46:03.001865 17377 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35836: token expired
W20260807 19:46:03.002434 16776 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.208.62:37421)
I20260807 19:46:03.015331 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.002906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35850 (local address 127.30.208.62:37421)
0807 19:46:03.003039 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:03.003043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:03.003058 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:46:03.003166 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:03.003169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:03.003223 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:03.003294 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:03.004601 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.005399 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.006835 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.007299 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.007393 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.009997 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:03.010002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:03.010006 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:03.010038 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:03.012644 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:03.013584 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:03.013588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:03.013590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:03.013722 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:03.014146 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:03.014149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:03.014151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:03.014470 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:03.014663 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:03.014911 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 19:46:03.015076 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 19:46:03.018905 16867 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35850:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:46:03.026160 16867 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:46:03.060201 17385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.038573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.30.208.1:33293)
0807 19:46:03.050417 (+ 11844us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:03.050427 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:03.051984 (+ 1557us) server_negotiation.cc:407] Connection header received
0807 19:46:03.052222 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:03.052231 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:03.052475 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:03.052662 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:03.053332 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.054882 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.057718 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.058880 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.059023 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.059261 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:03.059713 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 19:46:03.059889 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11668,"thread_start_us":12427,"threads_started":1}
I20260807 19:46:03.060256 17384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.038413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:33293 (local address 127.0.0.1:38774)
0807 19:46:03.051726 (+ 13313us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:03.051763 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:03.051947 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:03.052714 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:03.052722 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:03.052750 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:03.053084 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:03.053095 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.055059 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.055065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:03.057560 (+ 2495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:03.057575 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.059135 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.059142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:03.059426 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.059490 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 19:46:03.059611 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 19:46:03.059814 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13089,"mutex_wait_us":21,"thread_start_us":12921,"threads_started":1}
I20260807 19:46:03.062403 17387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.039127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40208 (local address 127.30.208.2:34111)
0807 19:46:03.054059 (+ 14932us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:03.054067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:03.054093 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:46:03.054178 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:03.054184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:03.054412 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:03.054588 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:03.055302 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.056553 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.059566 (+ 3013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.061002 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.061144 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.061374 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:03.061892 (+ 518us) server_negotiation.cc:299] Negotiation successful
0807 19:46:03.062060 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14776,"thread_start_us":14660,"threads_started":1}
I20260807 19:46:03.063169 17386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.038546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:34111 (local address 127.0.0.1:40208)
0807 19:46:03.052418 (+ 13872us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:03.052436 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:03.052541 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:03.054612 (+ 2071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:03.054618 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:03.054648 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:03.055058 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:03.055078 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.056707 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.056711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:03.059397 (+ 2686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:03.059414 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.061536 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.061542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:03.061697 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.061730 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:46:03.062667 (+ 937us) client_negotiation.cc:240] Negotiation successful
0807 19:46:03.062804 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13753,"mutex_wait_us":74,"thread_start_us":13622,"threads_started":1}
I20260807 19:46:03.063814 17388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.052916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46944 (local address 127.30.208.3:43785)
0807 19:46:03.053612 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:03.053620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:03.055381 (+ 1761us) server_negotiation.cc:407] Connection header received
0807 19:46:03.055675 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:03.055682 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:03.055883 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:03.056038 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:03.057028 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.058662 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.061274 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.062632 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.062729 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.062943 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:03.063347 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 19:46:03.063500 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":331,"threads_started":1}
I20260807 19:46:03.064553 17389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.053584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:43785 (local address 127.0.0.1:46944)
0807 19:46:03.055258 (+ 1674us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:03.055275 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:03.055488 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:03.056358 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:03.056362 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:03.056391 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:03.056823 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:03.056837 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.058818 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.058824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:03.061067 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:03.061081 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.062953 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.062960 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:03.063128 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.063162 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:46:03.064068 (+ 906us) client_negotiation.cc:240] Negotiation successful
0807 19:46:03.064199 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1542,"mutex_wait_us":29,"thread_start_us":2203,"threads_started":1}
I20260807 19:46:03.064689 17002 tablet_service.cc:1467] Processing CreateTablet for tablet c9ad51a36cec4f8d89e76f75515424ac (DEFAULT_TABLE table=test-table [id=8f2e6c5e68b741d3934a51a5bf933f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:03.066128 17002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ad51a36cec4f8d89e76f75515424ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:03.066339 17150 tablet_service.cc:1467] Processing CreateTablet for tablet c9ad51a36cec4f8d89e76f75515424ac (DEFAULT_TABLE table=test-table [id=8f2e6c5e68b741d3934a51a5bf933f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:03.067507 17150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ad51a36cec4f8d89e76f75515424ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:03.068245 17300 tablet_service.cc:1467] Processing CreateTablet for tablet c9ad51a36cec4f8d89e76f75515424ac (DEFAULT_TABLE table=test-table [id=8f2e6c5e68b741d3934a51a5bf933f85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:03.069677 17300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ad51a36cec4f8d89e76f75515424ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:03.079722 17390 tablet_bootstrap.cc:492] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676: Bootstrap starting.
I20260807 19:46:03.083058 17390 tablet_bootstrap.cc:654] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:03.084064 17390 log.cc:824] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:03.084985 17391 tablet_bootstrap.cc:492] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3: Bootstrap starting.
I20260807 19:46:03.087718 17075 heartbeater.cc:503] Master 127.30.208.62:37421 was elected leader, sending a full tablet report...
I20260807 19:46:03.089211 17391 tablet_bootstrap.cc:654] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:03.090788 17391 log.cc:824] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:03.091477 17393 tablet_bootstrap.cc:492] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094: Bootstrap starting.
I20260807 19:46:03.094034 17390 tablet_bootstrap.cc:492] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676: No bootstrap required, opened a new log
I20260807 19:46:03.094318 17390 ts_tablet_manager.cc:1397] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.008s
I20260807 19:46:03.095821 17393 tablet_bootstrap.cc:654] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:03.095891 17391 tablet_bootstrap.cc:492] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3: No bootstrap required, opened a new log
I20260807 19:46:03.096179 17391 ts_tablet_manager.cc:1397] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.006s
I20260807 19:46:03.096822 17393 log.cc:824] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:03.099200 17393 tablet_bootstrap.cc:492] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094: No bootstrap required, opened a new log
I20260807 19:46:03.099427 17393 ts_tablet_manager.cc:1397] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260807 19:46:03.104817 17390 raft_consensus.cc:348] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:03.105362 17390 raft_consensus.cc:374] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:03.105501 17390 raft_consensus.cc:729] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63a96326893847a5a6f3c63e031a9676, State: Initialized, Role: FOLLOWER
I20260807 19:46:03.106199 17390 consensus_queue.cc:260] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:03.106480 17391 raft_consensus.cc:348] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:03.107000 17391 raft_consensus.cc:374] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:03.107138 17391 raft_consensus.cc:729] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a9fb75e5e0340da8af60d54d2c3d4c3, State: Initialized, Role: FOLLOWER
I20260807 19:46:03.107508 17227 heartbeater.cc:503] Master 127.30.208.62:37421 was elected leader, sending a full tablet report...
I20260807 19:46:03.107394 17393 raft_consensus.cc:348] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:03.107827 17393 raft_consensus.cc:374] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:03.107926 17393 raft_consensus.cc:729] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d13b1d1b474bdaa50f36bc26b7c094, State: Initialized, Role: FOLLOWER
I20260807 19:46:03.107856 17391 consensus_queue.cc:260] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:03.108229 17390 ts_tablet_manager.cc:1428] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260807 19:46:03.108575 17393 consensus_queue.cc:260] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:03.109797 17373 heartbeater.cc:503] Master 127.30.208.62:37421 was elected leader, sending a full tablet report...
I20260807 19:46:03.112094 17393 ts_tablet_manager.cc:1428] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094: Time spent starting tablet: real 0.013s user 0.006s sys 0.005s
I20260807 19:46:03.113821 17391 ts_tablet_manager.cc:1428] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3: Time spent starting tablet: real 0.018s user 0.015s sys 0.002s
W20260807 19:46:03.212059 17374 tablet.cc:2369] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:46:03.256286 17228 tablet.cc:2369] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:46:03.298321 17076 tablet.cc:2369] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:46:03.351653 17398 raft_consensus.cc:482] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:03.351945 17398 raft_consensus.cc:504] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:03.353693 17398 leader_election.cc:288] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111), a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785)
I20260807 19:46:03.361069 17387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.354230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:49421 (local address 127.30.208.2:34111)
0807 19:46:03.354470 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:03.354478 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:03.354900 (+ 422us) server_negotiation.cc:407] Connection header received
0807 19:46:03.355152 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:03.355159 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:03.355238 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:03.355306 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:03.356107 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.357001 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.359135 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.360013 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.360137 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.360194 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:03.360605 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 19:46:03.360803 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:46:03.361196 17388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.355183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:36903 (local address 127.30.208.3:43785)
0807 19:46:03.355377 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:03.355383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:03.355546 (+ 163us) server_negotiation.cc:407] Connection header received
0807 19:46:03.355695 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:03.355700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:03.355771 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:03.355869 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:03.356498 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.357384 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.359361 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.360289 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.360447 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.360516 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:03.360811 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 19:46:03.360943 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:46:03.361071 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.354112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:34111 (local address 127.30.208.1:49421)
0807 19:46:03.354762 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:03.354779 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:03.354957 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:03.355504 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:03.355508 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:03.355536 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:03.355950 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:03.355968 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.357156 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.357162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:03.358962 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:03.358975 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.360247 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.360254 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:03.360422 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.360463 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 19:46:03.360559 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 19:46:03.360659 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":1,"thread_start_us":350,"threads_started":1}
I20260807 19:46:03.361382 17403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.354875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:43785 (local address 127.30.208.1:36903)
0807 19:46:03.355453 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:03.355467 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:03.355574 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:03.355896 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:03.355899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:03.355917 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:03.356298 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:03.356311 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.357540 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.357545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:03.359183 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:03.359207 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.360486 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.360491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:03.360649 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.360682 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:46:03.360775 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 19:46:03.360891 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":20,"thread_start_us":272,"threads_started":1}
I20260807 19:46:03.362358 17175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ad51a36cec4f8d89e76f75515424ac" candidate_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a96326893847a5a6f3c63e031a9676" is_pre_election: true
I20260807 19:46:03.362356 17325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ad51a36cec4f8d89e76f75515424ac" candidate_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" is_pre_election: true
I20260807 19:46:03.362814 17325 raft_consensus.cc:2454] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a9fb75e5e0340da8af60d54d2c3d4c3 in term 0.
I20260807 19:46:03.362819 17175 raft_consensus.cc:2454] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a9fb75e5e0340da8af60d54d2c3d4c3 in term 0.
I20260807 19:46:03.363422 16949 leader_election.cc:302] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a9fb75e5e0340da8af60d54d2c3d4c3, a1d13b1d1b474bdaa50f36bc26b7c094; no voters:
I20260807 19:46:03.363803 17398 raft_consensus.cc:2781] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:46:03.363948 17398 raft_consensus.cc:482] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:46:03.364037 17398 raft_consensus.cc:3037] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:03.366297 17398 raft_consensus.cc:504] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:03.366948 17398 leader_election.cc:288] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111), a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785)
I20260807 19:46:03.367400 17175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ad51a36cec4f8d89e76f75515424ac" candidate_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63a96326893847a5a6f3c63e031a9676"
I20260807 19:46:03.367486 17325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ad51a36cec4f8d89e76f75515424ac" candidate_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094"
I20260807 19:46:03.367586 17175 raft_consensus.cc:3037] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:03.367677 17325 raft_consensus.cc:3037] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:03.369902 17175 raft_consensus.cc:2454] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a9fb75e5e0340da8af60d54d2c3d4c3 in term 1.
I20260807 19:46:03.369921 17325 raft_consensus.cc:2454] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a9fb75e5e0340da8af60d54d2c3d4c3 in term 1.
I20260807 19:46:03.370304 16949 leader_election.cc:302] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a9fb75e5e0340da8af60d54d2c3d4c3, a1d13b1d1b474bdaa50f36bc26b7c094; no voters:
I20260807 19:46:03.370584 17398 raft_consensus.cc:2781] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:46:03.371104 17398 raft_consensus.cc:686] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 LEADER]: Becoming Leader. State: Replica: 6a9fb75e5e0340da8af60d54d2c3d4c3, State: Running, Role: LEADER
I20260807 19:46:03.371678 17398 consensus_queue.cc:237] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:03.376044 16867 catalog_manager.cc:5495] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a9fb75e5e0340da8af60d54d2c3d4c3 (127.30.208.1). New cstate: current_term: 1 leader_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } health_report { overall_health: UNKNOWN } } }
I20260807 19:46:03.416074 17385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.410962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38780 (local address 127.30.208.1:33293)
0807 19:46:03.411141 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:03.411146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:03.411165 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:46:03.411340 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:03.411344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:03.411432 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:03.411556 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:46:03.411889 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.412627 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.414009 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.414425 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.414517 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.415022 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:46:03.415542 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:46:03.415648 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:03.415796 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 19:46:03.415863 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:46:03.424000 17325 raft_consensus.cc:1261] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094 [term 1 FOLLOWER]: Refusing update from remote peer 6a9fb75e5e0340da8af60d54d2c3d4c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:46:03.424052 17175 raft_consensus.cc:1261] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676 [term 1 FOLLOWER]: Refusing update from remote peer 6a9fb75e5e0340da8af60d54d2c3d4c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:46:03.424985 17404 consensus_queue.cc:1035] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:46:03.425418 17398 consensus_queue.cc:1035] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:46:03.440647 17408 mvcc.cc:205] Tried to move back new op lower bound from 7315996522175471616 to 7315996521973968896. Current Snapshot: MvccSnapshot[applied={T|T < 7315996522175471616}]
I20260807 19:46:03.450330 16866 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:46:03.450666 16866 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35850:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:46:03.454742 16866 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Sending DeleteTablet for 3 replicas of tablet c9ad51a36cec4f8d89e76f75515424ac
I20260807 19:46:03.455966 17002 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ad51a36cec4f8d89e76f75515424ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:03 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:38774
I20260807 19:46:03.456096 17150 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ad51a36cec4f8d89e76f75515424ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:03 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:40208
I20260807 19:46:03.456522 17300 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ad51a36cec4f8d89e76f75515424ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:03 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:46944
I20260807 19:46:03.457610 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 16908
I20260807 19:46:03.468619 17418 tablet_replica.cc:330] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094: stopping tablet replica
I20260807 19:46:03.469333 17418 raft_consensus.cc:2229] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:46:03.470073 17418 raft_consensus.cc:2258] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:03.472579 17418 ts_tablet_manager.cc:1905] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:03.477010 17417 tablet_replica.cc:330] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676: stopping tablet replica
I20260807 19:46:03.477571 17417 raft_consensus.cc:2229] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:46:03.478315 17417 raft_consensus.cc:2258] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:03.480757 17417 ts_tablet_manager.cc:1905] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:03.480860 17418 ts_tablet_manager.cc:1918] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:03.481024 17418 log.cc:1196] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/c9ad51a36cec4f8d89e76f75515424ac
I20260807 19:46:03.481567 17418 ts_tablet_manager.cc:1939] T c9ad51a36cec4f8d89e76f75515424ac P a1d13b1d1b474bdaa50f36bc26b7c094: Deleting consensus metadata
I20260807 19:46:03.483127 17385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.458519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38782 (local address 127.30.208.1:33293)
0807 19:46:03.458721 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:03.458728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:03.458982 (+ 254us) server_negotiation.cc:407] Connection header received
0807 19:46:03.459050 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:03.459055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:03.459137 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:03.459263 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:03.461759 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.463010 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.465024 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.465776 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.465902 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.471918 (+ 6016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:03.471934 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:03.471953 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:03.472010 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:03.476235 (+ 4225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:03.478262 (+ 2027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:03.478272 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:03.478276 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:03.478480 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:03.479369 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:03.479375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:03.479378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:03.479857 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:03.480154 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:03.481731 (+ 1577us) server_negotiation.cc:299] Negotiation successful
0807 19:46:03.482747 (+ 1016us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:46:03.483501 16851 catalog_manager.cc:4841] TS a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785): tablet c9ad51a36cec4f8d89e76f75515424ac (table test-table [id=8f2e6c5e68b741d3934a51a5bf933f85]) successfully deleted
I20260807 19:46:03.484066 17070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:03.489027 17417 ts_tablet_manager.cc:1918] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:03.489224 17417 log.cc:1196] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/c9ad51a36cec4f8d89e76f75515424ac
I20260807 19:46:03.489655 17417 ts_tablet_manager.cc:1939] T c9ad51a36cec4f8d89e76f75515424ac P 63a96326893847a5a6f3c63e031a9676: Deleting consensus metadata
I20260807 19:46:03.491158 16851 catalog_manager.cc:4841] TS 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111): tablet c9ad51a36cec4f8d89e76f75515424ac (table test-table [id=8f2e6c5e68b741d3934a51a5bf933f85]) successfully deleted
I20260807 19:46:03.597765 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 16908
W20260807 19:46:03.616223 16853 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.208.1:33293 (error 108)
I20260807 19:46:03.616966 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 17079
I20260807 19:46:03.618309 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.617441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:33293 (local address 127.0.0.1:38798)
0807 19:46:03.617936 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:03.618074 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:33293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":255,"threads_started":1}
W20260807 19:46:03.618570 16853 catalog_manager.cc:4584] TS 6a9fb75e5e0340da8af60d54d2c3d4c3 (127.30.208.1:33293): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c9ad51a36cec4f8d89e76f75515424ac: Network error: Client connection negotiation failed: client connection to 127.30.208.1:33293: connect: Connection refused (error 111)
I20260807 19:46:03.630722 17387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.617752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.30.208.2:34111)
0807 19:46:03.617925 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:03.617929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:03.618066 (+ 137us) server_negotiation.cc:407] Connection header received
0807 19:46:03.618237 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:03.618240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:03.618314 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:03.618411 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:03.619835 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.620528 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.621873 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.622272 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.622386 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.624978 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:03.624995 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:03.625011 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:03.625047 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:03.627818 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:03.628517 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:03.628524 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:03.628526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:03.628658 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:03.629060 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:03.629064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:03.629066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:03.629506 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:03.629790 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:03.630139 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 19:46:03.630377 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:46:03.631340 17222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:03.672111 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.671544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:33293 (local address 127.0.0.1:38804)
0807 19:46:03.671809 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:03.671936 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:33293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 19:46:03.673369 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.672889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:33293 (local address 127.0.0.1:38806)
0807 19:46:03.673063 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:03.673154 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:33293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 19:46:03.725279 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 17079
I20260807 19:46:03.734159 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.733681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:33293 (local address 127.0.0.1:38822)
0807 19:46:03.733910 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:03.734007 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:33293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 19:46:03.735316 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.734948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:33293 (local address 127.0.0.1:38836)
0807 19:46:03.735087 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:03.735159 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:33293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 19:46:03.743996 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 17230
I20260807 19:46:03.757481 17388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.744776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46946 (local address 127.30.208.3:43785)
0807 19:46:03.744936 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:03.744941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:03.744960 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:46:03.745053 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:03.745057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:03.745121 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:03.745245 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:03.746567 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.747356 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.748561 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:03.748972 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:03.749077 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:03.751646 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:03.751663 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:03.751679 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:03.751718 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:03.754366 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:03.755142 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:03.755150 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:03.755153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:03.755332 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:03.755799 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:03.755804 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:03.755806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:03.756242 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:03.756497 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:03.756866 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 19:46:03.757074 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:46:03.758159 17368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:03.815927 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.815436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:33293 (local address 127.0.0.1:38840)
0807 19:46:03.815656 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:03.815772 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:33293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 19:46:03.817101 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:03.816672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:33293 (local address 127.0.0.1:38844)
0807 19:46:03.816887 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:03.816960 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:33293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260807 19:46:03.850620 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 17230
I20260807 19:46:03.868765 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 16828
I20260807 19:46:03.869983 16901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:03.986344 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 16828
I20260807 19:46:24.010207 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:37421
--webserver_interface=127.30.208.62
--webserver_port=40599
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:37421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:24.358608 17425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:24.358948 17425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:24.359088 17425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:24.366537 17425 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 19:46:24.366653 17425 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 19:46:24.366710 17425 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 19:46:24.366781 17425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:46:24.366859 17425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:24.366966 17425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:24.367023 17425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:46:24.367098 17425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:46:24.377660 17425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:37421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:37421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=40599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:24.379107 17425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:24.380539 17425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:24.393473 17430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:24.393518 17431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:24.393494 17433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:24.394198 17425 server_base.cc:1034] running on GCE node
I20260807 19:46:24.395083 17425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:24.396592 17425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:24.397838 17425 hybrid_clock.cc:648] HybridClock initialized: now 1786131984397763 us; error 39 us; skew 500 ppm
I20260807 19:46:24.398227 17425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:24 dist-test-slave-vq2q krb5kdc[16205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131984, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:24.401612 17425 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:46:24.403136 17425 webserver.cc:460] Webserver started at http://127.30.208.62:40599/ using document root <none> and password file <none>
I20260807 19:46:24.403839 17425 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:24.403945 17425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:24.409684 17425 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:46:24.412837 17441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:24.414378 17425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:46:24.414543 17425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "13947c19c459464cb19d9bf29abdf38d"
format_stamp: "Formatted at 2026-08-07 19:45:38 on dist-test-slave-vq2q"
server_key: "2cb9b4ea3254fe45dd34d9bd1e56471e"
server_key_iv: "854738b13ad05a167f7a5ca5bcbf0dcc"
server_key_version: "encryptionkey@0"
I20260807 19:46:24.415582 17425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:24.445097 17425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:24.449646 17425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:24.449947 17425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:24.468781 17425 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:37421
I20260807 19:46:24.468816 17500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:37421 every 8 connection(s)
I20260807 19:46:24.470356 17425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:46:24.473341 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 17425
I20260807 19:46:24.474409 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:33293
--local_ip_for_outbound_sockets=127.30.208.1
--tserver_master_addrs=127.30.208.62:37421
--webserver_port=44953
--webserver_interface=127.30.208.1
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:46:24.479576 17501 sys_catalog.cc:263] Verifying existing consensus state
I20260807 19:46:24.484091 17501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Bootstrap starting.
I20260807 19:46:24.506263 17501 log.cc:824] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:24.528903 17501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:46:24.529697 17501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Bootstrap complete.
I20260807 19:46:24.544062 17501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } }
I20260807 19:46:24.545637 17501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13947c19c459464cb19d9bf29abdf38d, State: Initialized, Role: FOLLOWER
I20260807 19:46:24.546485 17501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } }
I20260807 19:46:24.546792 17501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:46:24.546928 17501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:46:24.547115 17501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 2 FOLLOWER]: Advancing to term 3
I20260807 19:46:24.552927 17501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } }
I20260807 19:46:24.553580 17501 leader_election.cc:302] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13947c19c459464cb19d9bf29abdf38d; no voters:
I20260807 19:46:24.554309 17501 leader_election.cc:288] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 19:46:24.554459 17505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 3 FOLLOWER]: Leader election won for term 3
I20260807 19:46:24.555639 17505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [term 3 LEADER]: Becoming Leader. State: Replica: 13947c19c459464cb19d9bf29abdf38d, State: Running, Role: LEADER
I20260807 19:46:24.556170 17505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } }
I20260807 19:46:24.556546 17501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:46:24.558583 17507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13947c19c459464cb19d9bf29abdf38d. Latest consensus state: current_term: 3 leader_uuid: "13947c19c459464cb19d9bf29abdf38d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } } }
I20260807 19:46:24.558982 17507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:24.559585 17506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "13947c19c459464cb19d9bf29abdf38d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13947c19c459464cb19d9bf29abdf38d" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37421 } } }
I20260807 19:46:24.559882 17506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:24.569315 17511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:46:24.578091 17511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:46:24.579651 17511 catalog_manager.cc:1259] Loaded cluster ID: 3f960a5603ac46f9bc50465b12d6b1cf
I20260807 19:46:24.579818 17511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:46:24.583968 17511 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:46:24.585714 17511 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Loaded TSK: 0
I20260807 19:46:24.585956 17511 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 13947c19c459464cb19d9bf29abdf38d: Loaded TSK: 1
F20260807 19:46:24.587188 17511 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbf1bd169ad google::LogMessage::Fail() at ??:0
@ 0x7fbf1bd1988a google::LogMessage::SendToLog() at ??:0
@ 0x7fbf1bd164de google::LogMessage::Flush() at ??:0
@ 0x7fbf1bd17a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
I20260807 19:46:24.597470 17525 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fbf1fa7a7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fbf1fa6973e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fbf317451c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fbf31744783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fbf317d02a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fbf317d00c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fbf3181a5e9 std::function<>::operator()() at ??:0
@ 0x7fbf31748cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fbf31746c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbf317ce610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fbf317ce439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fbf1e962345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbf1e96591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fbf1e965799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fbf1e940eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbf21c1a6db start_thread at ??:0
@ 0x7fbf1abb061f clone at ??:0
W20260807 19:46:24.911216 17503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:24.911525 17503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:24.911625 17503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:24.919229 17503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:24.919353 17503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:24.919476 17503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:46:24.929682 17503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:33293
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=44953
--tserver_master_addrs=127.30.208.62:37421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:24.930974 17503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:24.932310 17503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:24.945809 17553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:24.945935 17554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:24.946075 17557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:24.946914 17503 server_base.cc:1034] running on GCE node
I20260807 19:46:24.947611 17503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:24.948556 17503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:24.949770 17503 hybrid_clock.cc:648] HybridClock initialized: now 1786131984949709 us; error 57 us; skew 500 ppm
I20260807 19:46:24.950078 17503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:24 dist-test-slave-vq2q krb5kdc[16205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131984, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:24.953222 17503 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:46:24.954633 17503 webserver.cc:460] Webserver started at http://127.30.208.1:44953/ using document root <none> and password file <none>
I20260807 19:46:24.955312 17503 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:24.955457 17503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:24.960800 17503 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:46:24.963611 17565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:24.964881 17503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 19:46:24.965027 17503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3"
format_stamp: "Formatted at 2026-08-07 19:45:39 on dist-test-slave-vq2q"
server_key: "386b718c59f96658447e588648199cb2"
server_key_iv: "82154591a58503a697620253f1c91702"
server_key_version: "encryptionkey@0"
I20260807 19:46:24.966284 17503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:24.980324 17503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:24.984536 17503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:24.984802 17503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:24.985839 17503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:24.987799 17572 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260807 19:46:24.993767 17503 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 19:46:24.993916 17503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260807 19:46:24.994059 17503 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 19:46:24.997359 17503 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 19:46:24.997449 17503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.004s sys 0.000s
I20260807 19:46:24.997884 17572 tablet_bootstrap.cc:492] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3: Bootstrap starting.
I20260807 19:46:25.025902 17572 log.cc:824] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:25.035288 17503 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:33293
I20260807 19:46:25.035439 17688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:33293 every 8 connection(s)
I20260807 19:46:25.036768 17503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:46:25.039426 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 17503
I20260807 19:46:25.040009 17572 tablet_bootstrap.cc:492] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 19:46:25.040220 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:34111
--local_ip_for_outbound_sockets=127.30.208.2
--tserver_master_addrs=127.30.208.62:37421
--webserver_port=45767
--webserver_interface=127.30.208.2
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:46:25.040671 17572 tablet_bootstrap.cc:492] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3: Bootstrap complete.
I20260807 19:46:25.041944 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:25.039497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:45045)
0807 19:46:25.040704 (+ 1207us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:25.041023 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":424,"threads_started":1}
I20260807 19:46:25.042332 17572 ts_tablet_manager.cc:1397] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3: Time spent bootstrapping tablet: real 0.045s user 0.028s sys 0.015s
W20260807 19:46:25.042603 17689 heartbeater.cc:644] Failed to heartbeat to 127.30.208.62:37421 (0 consecutive failures): Network error: Failed to ping master at 127.30.208.62:37421: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
I20260807 19:46:25.043509 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:25.043159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:35245)
0807 19:46:25.043317 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:25.043389 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 19:46:25.044445 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:25.044050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:55991)
0807 19:46:25.044205 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:25.044300 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 19:46:25.044734 17689 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 19:46:25.051016 17572 raft_consensus.cc:348] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:25.052201 17572 raft_consensus.cc:729] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a9fb75e5e0340da8af60d54d2c3d4c3, State: Initialized, Role: FOLLOWER
I20260807 19:46:25.052868 17572 consensus_queue.cc:260] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:25.054539 17572 ts_tablet_manager.cc:1428] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260807 19:46:25.054772 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:25.054299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:44053)
0807 19:46:25.054532 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:25.054619 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260807 19:46:25.055902 17691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:25.055523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:51291)
0807 19:46:25.055674 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:25.055755 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 19:46:25.378471 17692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:25.378749 17692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:25.378855 17692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:25.386175 17692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:25.386299 17692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:25.386420 17692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:46:25.396078 17692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:34111
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=45767
--tserver_master_addrs=127.30.208.62:37421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:25.397341 17692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:25.398644 17692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:25.411521 17698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:25.411491 17701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:25.411552 17699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:25.412575 17692 server_base.cc:1034] running on GCE node
I20260807 19:46:25.413229 17692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:25.414309 17692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:25.415548 17692 hybrid_clock.cc:648] HybridClock initialized: now 1786131985415484 us; error 64 us; skew 500 ppm
I20260807 19:46:25.415895 17692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:25 dist-test-slave-vq2q krb5kdc[16205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131985, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:25.419045 17692 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:46:25.420521 17692 webserver.cc:460] Webserver started at http://127.30.208.2:45767/ using document root <none> and password file <none>
I20260807 19:46:25.421221 17692 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:25.421352 17692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:25.426695 17692 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260807 19:46:25.429517 17709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:25.430819 17692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:46:25.430974 17692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "63a96326893847a5a6f3c63e031a9676"
format_stamp: "Formatted at 2026-08-07 19:45:39 on dist-test-slave-vq2q"
server_key: "e40ccb92e33fdef1e76ececb236d9fce"
server_key_iv: "7e33bcff8596ed96dd971598214832fd"
server_key_version: "encryptionkey@0"
I20260807 19:46:25.431969 17692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:25.462294 17692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:25.466699 17692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:25.466974 17692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:25.467957 17692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:25.469333 17692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:25.469429 17692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:25.469534 17692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:25.469604 17692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:25.502379 17692 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:34111
I20260807 19:46:25.502434 17829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:34111 every 8 connection(s)
I20260807 19:46:25.503854 17692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:46:25.508735 17832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:25.506814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:42137)
0807 19:46:25.507542 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:25.507856 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":275,"threads_started":1}
W20260807 19:46:25.509469 17830 heartbeater.cc:644] Failed to heartbeat to 127.30.208.62:37421 (0 consecutive failures): Network error: Failed to ping master at 127.30.208.62:37421: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
I20260807 19:46:25.510396 17832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:25.510016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:37063)
0807 19:46:25.510202 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:25.510278 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260807 19:46:25.511262 17832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:25.510926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:54677)
0807 19:46:25.511065 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:25.511144 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 19:46:25.511493 17830 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 19:46:25.512535 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 17692
I20260807 19:46:25.513257 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:43785
--local_ip_for_outbound_sockets=127.30.208.3
--tserver_master_addrs=127.30.208.62:37421
--webserver_port=38727
--webserver_interface=127.30.208.3
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:25.847910 17833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:25.848217 17833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:25.848344 17833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:25.855796 17833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:25.855926 17833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:25.856096 17833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:46:25.866145 17833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:43785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=38727
--tserver_master_addrs=127.30.208.62:37421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:25.867513 17833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:25.868845 17833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:25.881820 17839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:25.881914 17840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:25.881894 17842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:25.882905 17833 server_base.cc:1034] running on GCE node
I20260807 19:46:25.883518 17833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:25.884459 17833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:25.885659 17833 hybrid_clock.cc:648] HybridClock initialized: now 1786131985885603 us; error 48 us; skew 500 ppm
I20260807 19:46:25.885986 17833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:25 dist-test-slave-vq2q krb5kdc[16205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131985, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:25.889048 17833 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:46:25.890547 17833 webserver.cc:460] Webserver started at http://127.30.208.3:38727/ using document root <none> and password file <none>
I20260807 19:46:25.891222 17833 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:25.891328 17833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:25.896775 17833 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 19:46:25.899577 17849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:25.900919 17833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 19:46:25.901075 17833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "a1d13b1d1b474bdaa50f36bc26b7c094"
format_stamp: "Formatted at 2026-08-07 19:45:40 on dist-test-slave-vq2q"
server_key: "c76679038d74e7d7321ca77ba429b52f"
server_key_iv: "17fe3f427c21ac757dd322255e00e7c5"
server_key_version: "encryptionkey@0"
I20260807 19:46:25.902107 17833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:25.926302 17833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:25.931133 17833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:25.931397 17833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:25.932451 17833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:25.933911 17833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:25.934005 17833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:25.934087 17833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:25.934140 17833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:25.964879 17833 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:43785
I20260807 19:46:25.964932 17973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:43785 every 8 connection(s)
I20260807 19:46:25.966347 17833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:46:25.971865 17976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:25.969449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:59219)
0807 19:46:25.970578 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:25.970846 (+ 268us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":569,"threads_started":1}
I20260807 19:46:25.972035 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 17833
W20260807 19:46:25.972481 17974 heartbeater.cc:644] Failed to heartbeat to 127.30.208.62:37421 (0 consecutive failures): Network error: Failed to ping master at 127.30.208.62:37421: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
I20260807 19:46:25.973425 17976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:25.972999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:42399)
0807 19:46:25.973164 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:25.973280 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 19:46:25.974391 17976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:25.974059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:52989)
0807 19:46:25.974189 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:25.974266 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 19:46:25.974593 17974 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 19:46:26.057708 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:26.056901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:59549)
0807 19:46:26.057406 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:26.057508 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":227,"threads_started":1}
I20260807 19:46:26.512838 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:26.512032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:56531)
0807 19:46:26.512516 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:26.512607 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":207,"threads_started":1}
I20260807 19:46:26.539834 17980 raft_consensus.cc:482] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:26.540091 17980 raft_consensus.cc:504] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:26.541673 17980 leader_election.cc:288] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111), a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785)
Aug 07 19:46:26 dist-test-slave-vq2q krb5kdc[16205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131984, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
Aug 07 19:46:26 dist-test-slave-vq2q krb5kdc[16205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786131984, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:46:26.565687 17981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:26.542160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:45103 (local address 127.30.208.2:34111)
0807 19:46:26.543970 (+ 1810us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:26.543986 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:26.544033 (+ 47us) server_negotiation.cc:407] Connection header received
0807 19:46:26.546309 (+ 2276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:26.546322 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:26.546811 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:26.547185 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:26.549756 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:26.551620 (+ 1864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:26.554208 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:26.555220 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:26.555410 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:26.559081 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:26.559092 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:26.559111 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:26.559157 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:26.562026 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:26.562869 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:26.562876 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:26.562880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:26.563028 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:26.563483 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:26.563486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:26.563488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:26.563973 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:26.564143 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:26.564902 (+ 759us) server_negotiation.cc:299] Negotiation successful
0807 19:46:26.565308 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1619,"thread_start_us":1446,"threads_started":1}
I20260807 19:46:26.565701 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:26.542491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:43785 (local address 127.30.208.1:38045)
0807 19:46:26.543592 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:26.543611 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:26.543725 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:26.547364 (+ 3639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:26.547376 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:26.548394 (+ 1018us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:26.549504 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:26.549534 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:26.551838 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:26.551843 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:26.554051 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:26.554067 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:26.555702 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:26.555714 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:26.555844 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:26.556451 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:26.556468 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:26.558996 (+ 2528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:26.562245 (+ 3249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:26.562249 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:26.562251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:26.562747 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:26.563169 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:26.563173 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:26.563175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:26.563365 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:26.564107 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:26.564115 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:26.564408 (+ 293us) client_negotiation.cc:769] Sending connection context
0807 19:46:26.564641 (+ 233us) client_negotiation.cc:240] Negotiation successful
0807 19:46:26.565199 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"mutex_wait_us":220,"spinlock_wait_cycles":20096,"thread_start_us":824,"threads_started":1}
I20260807 19:46:26.565716 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:26.542871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:38045 (local address 127.30.208.3:43785)
0807 19:46:26.544229 (+ 1358us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:26.544242 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:26.544278 (+ 36us) server_negotiation.cc:407] Connection header received
0807 19:46:26.546309 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:26.546321 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:26.546811 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:26.547185 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:26.549730 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:26.551620 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:26.554229 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:26.555227 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:26.555404 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:26.559233 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:26.559240 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:26.559251 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:26.559292 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:26.562109 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:26.562888 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:26.562894 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:26.562899 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:26.563055 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:26.563480 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:26.563484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:26.563486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:26.563967 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:26.564132 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:26.564914 (+ 782us) server_negotiation.cc:299] Negotiation successful
0807 19:46:26.565277 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1178,"thread_start_us":1056,"threads_started":1}
I20260807 19:46:26.566311 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:26.542062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:34111 (local address 127.30.208.1:45103)
0807 19:46:26.542262 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:26.542302 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:26.542848 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:26.547364 (+ 4516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:26.547376 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:26.548394 (+ 1018us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:26.549504 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:26.549534 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:26.551840 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:26.551845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:26.554051 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:26.554067 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:26.555413 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:26.555419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:26.555613 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:26.556321 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:26.556354 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:26.558752 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:26.562182 (+ 3430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:26.562192 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:26.562206 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:26.562726 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:26.563144 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:26.563149 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:26.563152 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:26.563348 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:26.564111 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:26.564118 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:26.564426 (+ 308us) client_negotiation.cc:769] Sending connection context
0807 19:46:26.564640 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 19:46:26.565353 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":600,"spinlock_wait_cycles":36608}
W20260807 19:46:26.568939 17567 leader_election.cc:341] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111): Not found: Tablet not found: c9ad51a36cec4f8d89e76f75515424ac
W20260807 19:46:26.569257 17567 leader_election.cc:341] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785): Not found: Tablet not found: c9ad51a36cec4f8d89e76f75515424ac
I20260807 19:46:26.569377 17567 leader_election.cc:302] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a9fb75e5e0340da8af60d54d2c3d4c3; no voters: 63a96326893847a5a6f3c63e031a9676, a1d13b1d1b474bdaa50f36bc26b7c094
I20260807 19:46:26.569774 17980 raft_consensus.cc:2726] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:46:26.976023 17984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:26.975152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:36539)
0807 19:46:26.975640 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:26.975795 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":244,"threads_started":1}
I20260807 19:46:27.059396 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:27.058830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:39007)
0807 19:46:27.059060 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:27.059172 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 19:46:27.514494 17985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:27.513759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:59505)
0807 19:46:27.514231 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:27.514322 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":248,"threads_started":1}
I20260807 19:46:27.977695 17986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:27.976908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:60327)
0807 19:46:27.977390 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:27.977493 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":260,"threads_started":1}
I20260807 19:46:28.061237 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:28.060344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:56023)
0807 19:46:28.060928 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:28.061021 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":234,"threads_started":1}
I20260807 19:46:28.203492 17988 raft_consensus.cc:482] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:28.203672 17988 raft_consensus.cc:504] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:28.204397 17988 leader_election.cc:288] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111), a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785)
W20260807 19:46:28.205442 17567 leader_election.cc:341] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111): Not found: Tablet not found: c9ad51a36cec4f8d89e76f75515424ac
W20260807 19:46:28.205682 17567 leader_election.cc:341] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785): Not found: Tablet not found: c9ad51a36cec4f8d89e76f75515424ac
I20260807 19:46:28.205771 17567 leader_election.cc:302] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a9fb75e5e0340da8af60d54d2c3d4c3; no voters: 63a96326893847a5a6f3c63e031a9676, a1d13b1d1b474bdaa50f36bc26b7c094
I20260807 19:46:28.206053 17988 raft_consensus.cc:2726] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:46:28.516091 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:28.515382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:60907)
0807 19:46:28.515829 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:28.515918 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":235,"threads_started":1}
I20260807 19:46:28.979641 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:28.978751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:55535)
0807 19:46:28.979295 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:28.979423 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":274,"threads_started":1}
I20260807 19:46:29.062743 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:29.062073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:44797)
0807 19:46:29.062499 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:29.062583 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":222,"threads_started":1}
I20260807 19:46:29.517889 17993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:29.517059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:53621)
0807 19:46:29.517587 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:29.517693 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":272,"threads_started":1}
I20260807 19:46:29.981463 17994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:29.980655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:58037)
0807 19:46:29.981125 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:29.981251 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":236,"threads_started":1}
I20260807 19:46:30.064947 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:30.064023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:38405)
0807 19:46:30.064635 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:30.064733 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":298,"threads_started":1}
I20260807 19:46:30.278918 17997 raft_consensus.cc:482] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:30.279161 17997 raft_consensus.cc:504] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:30.279910 17997 leader_election.cc:288] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111), a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785)
W20260807 19:46:30.280969 17567 leader_election.cc:341] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785): Not found: Tablet not found: c9ad51a36cec4f8d89e76f75515424ac
W20260807 19:46:30.281214 17567 leader_election.cc:341] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111): Not found: Tablet not found: c9ad51a36cec4f8d89e76f75515424ac
I20260807 19:46:30.281327 17567 leader_election.cc:302] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a9fb75e5e0340da8af60d54d2c3d4c3; no voters: 63a96326893847a5a6f3c63e031a9676, a1d13b1d1b474bdaa50f36bc26b7c094
I20260807 19:46:30.281581 17997 raft_consensus.cc:2726] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:46:30.519691 17998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:30.518880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:51499)
0807 19:46:30.519398 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:30.519514 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":243,"threads_started":1}
I20260807 19:46:30.983182 17999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:30.982446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:34089)
0807 19:46:30.982927 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:30.983010 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":259,"threads_started":1}
I20260807 19:46:31.066748 18000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:31.065950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:49517)
0807 19:46:31.066465 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:31.066570 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":280,"threads_started":1}
I20260807 19:46:31.521508 18001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:31.520669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:57215)
0807 19:46:31.521216 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:31.521320 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":279,"threads_started":1}
I20260807 19:46:31.984827 18002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:31.984079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:51521)
0807 19:46:31.984553 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:31.984639 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":256,"threads_started":1}
I20260807 19:46:32.068467 18003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:32.067696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:57831)
0807 19:46:32.068179 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:32.068264 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":268,"threads_started":1}
I20260807 19:46:32.523344 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:32.522466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:39367)
0807 19:46:32.523030 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:32.523131 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":251,"threads_started":1}
I20260807 19:46:32.986639 18005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:32.985754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:49837)
0807 19:46:32.986356 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:32.986445 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":250,"threads_started":1}
I20260807 19:46:33.070166 18006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:33.069426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:46905)
0807 19:46:33.069906 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:33.070005 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":230,"threads_started":1}
I20260807 19:46:33.525238 18007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:33.524351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:45287)
0807 19:46:33.524882 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:33.524994 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":236,"threads_started":1}
I20260807 19:46:33.660609 18008 raft_consensus.cc:482] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:33.660810 18008 raft_consensus.cc:504] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:33.661463 18008 leader_election.cc:288] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111), a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785)
W20260807 19:46:33.662499 17567 leader_election.cc:341] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111): Not found: Tablet not found: c9ad51a36cec4f8d89e76f75515424ac
W20260807 19:46:33.662745 17567 leader_election.cc:341] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785): Not found: Tablet not found: c9ad51a36cec4f8d89e76f75515424ac
I20260807 19:46:33.662839 17567 leader_election.cc:302] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a9fb75e5e0340da8af60d54d2c3d4c3; no voters: 63a96326893847a5a6f3c63e031a9676, a1d13b1d1b474bdaa50f36bc26b7c094
I20260807 19:46:33.663125 18008 raft_consensus.cc:2726] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:46:33.988276 18009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:33.987578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:39233)
0807 19:46:33.988003 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:33.988089 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":231,"threads_started":1}
I20260807 19:46:34.072036 18011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:34.071144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:48977)
0807 19:46:34.071714 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:34.071827 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":259,"threads_started":1}
I20260807 19:46:34.527072 18013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:34.526252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:35229)
0807 19:46:34.526802 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:34.526908 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":276,"threads_started":1}
I20260807 19:46:34.990134 18014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:34.989271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:56841)
0807 19:46:34.989862 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:34.989971 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":282,"threads_started":1}
I20260807 19:46:35.073770 18015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:35.073054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:39751)
0807 19:46:35.073506 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:35.073592 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":240,"threads_started":1}
I20260807 19:46:35.528753 18016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:35.528026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:58405)
0807 19:46:35.528489 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:35.528575 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":248,"threads_started":1}
I20260807 19:46:35.992008 18017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:35.991247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:46775)
0807 19:46:35.991725 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:35.991813 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":250,"threads_started":1}
I20260807 19:46:36.075547 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:36.074733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:48587)
0807 19:46:36.075260 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:36.075357 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":298,"threads_started":1}
I20260807 19:46:36.126820 18020 raft_consensus.cc:482] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:36.127010 18020 raft_consensus.cc:504] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a9fb75e5e0340da8af60d54d2c3d4c3" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 33293 } } peers { permanent_uuid: "63a96326893847a5a6f3c63e031a9676" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34111 } } peers { permanent_uuid: "a1d13b1d1b474bdaa50f36bc26b7c094" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 43785 } }
I20260807 19:46:36.127668 18020 leader_election.cc:288] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111), a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785)
W20260807 19:46:36.128746 17567 leader_election.cc:341] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 63a96326893847a5a6f3c63e031a9676 (127.30.208.2:34111): Not found: Tablet not found: c9ad51a36cec4f8d89e76f75515424ac
W20260807 19:46:36.128963 17567 leader_election.cc:341] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a1d13b1d1b474bdaa50f36bc26b7c094 (127.30.208.3:43785): Not found: Tablet not found: c9ad51a36cec4f8d89e76f75515424ac
I20260807 19:46:36.129056 17567 leader_election.cc:302] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a9fb75e5e0340da8af60d54d2c3d4c3; no voters: 63a96326893847a5a6f3c63e031a9676, a1d13b1d1b474bdaa50f36bc26b7c094
I20260807 19:46:36.129297 18020 raft_consensus.cc:2726] T c9ad51a36cec4f8d89e76f75515424ac P 6a9fb75e5e0340da8af60d54d2c3d4c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 19:46:36.530594 18021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:36.529812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:34027)
0807 19:46:36.530314 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:36.530422 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":234,"threads_started":1}
I20260807 19:46:36.993760 18022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:36.992948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:53567)
0807 19:46:36.993494 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:36.993581 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":294,"threads_started":1}
I20260807 19:46:37.077369 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:37.076505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:52323)
0807 19:46:37.077048 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:37.077144 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":304,"threads_started":1}
I20260807 19:46:37.532452 18024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:37.531699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:41869)
0807 19:46:37.532177 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:37.532262 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":266,"threads_started":1}
I20260807 19:46:37.995509 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:37.994747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:50995)
0807 19:46:37.995237 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:37.995331 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":259,"threads_started":1}
I20260807 19:46:38.079265 18027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:38.078406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:60433)
0807 19:46:38.078968 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:38.079085 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":309,"threads_started":1}
I20260807 19:46:38.534264 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:38.533509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:60093)
0807 19:46:38.533984 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:38.534089 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":241,"threads_started":1}
I20260807 19:46:38.997339 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:38.996493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:39449)
0807 19:46:38.997004 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:38.997103 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":272,"threads_started":1}
I20260807 19:46:39.081017 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:39.080207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:51333)
0807 19:46:39.080702 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:39.080794 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":269,"threads_started":1}
I20260807 19:46:39.535989 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:39.535214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:53099)
0807 19:46:39.535701 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:39.535790 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":237,"threads_started":1}
I20260807 19:46:39.999161 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:39.998274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:40221)
0807 19:46:39.998855 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:39.998958 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":297,"threads_started":1}
I20260807 19:46:40.083091 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:40.082270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.1:55715)
0807 19:46:40.082819 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:40.082915 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":256,"threads_started":1}
I20260807 19:46:40.537912 18035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:40.537080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.2:43155)
0807 19:46:40.537594 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:40.537676 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":271,"threads_started":1}
I20260807 19:46:40.983122 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 17503
I20260807 19:46:40.998623 18036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:40.984242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53712 (local address 127.30.208.1:33293)
0807 19:46:40.984922 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:40.984932 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:40.984972 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:46:40.985051 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:40.985060 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:40.985369 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:40.985503 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:40.987094 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:40.988101 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:40.989384 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:40.989976 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:40.990092 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:40.992651 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:40.992664 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:40.992679 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:40.992721 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:40.995568 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:40.996280 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:40.996287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:40.996289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:40.996446 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:40.996890 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:40.996894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:40.996896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:40.997362 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:40.997619 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:40.998042 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 19:46:40.998278 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":369,"threads_started":1}
I20260807 19:46:41.000041 17684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:41.001127 18039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:41.000319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37421 (local address 127.30.208.3:36647)
0807 19:46:41.000803 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:41.000934 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.62:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":239,"threads_started":1}
I20260807 19:46:41.090785 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 17503
I20260807 19:46:41.109282 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 17692
I20260807 19:46:41.122112 18041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:41.110015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57356 (local address 127.30.208.2:34111)
0807 19:46:41.110510 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:41.110515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:41.110533 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:46:41.110589 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:41.110593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:41.110652 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:41.110746 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:41.112074 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:41.112950 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:41.114210 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:41.114653 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:41.114771 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:41.117442 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:41.117447 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:41.117451 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:41.117485 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:41.119862 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:41.120496 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:41.120500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:41.120502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:41.120628 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:41.120916 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:41.120919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:41.120921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:41.121210 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:41.121411 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:41.121650 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 19:46:41.121816 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":286,"threads_started":1}
I20260807 19:46:41.122810 17824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:41.205391 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 17692
I20260807 19:46:41.222854 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 17833
I20260807 19:46:41.235816 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:41.223669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49920 (local address 127.30.208.3:43785)
0807 19:46:41.224243 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:41.224248 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:41.224266 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:46:41.224320 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:41.224331 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:41.224400 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:41.224488 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:41.225888 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:41.226667 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:41.227781 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:41.228187 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:41.228273 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:41.230796 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:41.230802 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:41.230805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:41.230838 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:41.233565 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:41.234168 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:41.234172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:41.234174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:41.234352 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:41.234632 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:41.234635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:41.234637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:41.234879 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:41.235089 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:41.235266 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 19:46:41.235449 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":305,"threads_started":1}
I20260807 19:46:41.236524 17969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:41.318984 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 17833
2026-08-07T19:46:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64575 ms)
[----------] 30 tests from SecurityITest (299240 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:46:41 dist-test-slave-vq2q krb5kdc[18047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:46:41 dist-test-slave-vq2q krb5kdc[18047](info): set up 2 sockets
Aug 07 19:46:41 dist-test-slave-vq2q krb5kdc[18047](info): commencing operation
krb5kdc: starting...
W20260807 19:46:42.409703 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.002s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:46:42 dist-test-slave-vq2q krb5kdc[18047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:46:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:46:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:46:42.701352 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:34941
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:40083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:34941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:43.045082 18061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:43.045406 18061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:43.045538 18061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:43.053139 18061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:46:43.053300 18061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:43.053383 18061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:43.053468 18061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:46:43.053529 18061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:46:43.063652 18061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:40083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:34941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:34941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:43.064975 18061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:43.066319 18061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:43.078191 18068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:43.078286 18067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:43.078224 18070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:43.078675 18061 server_base.cc:1034] running on GCE node
I20260807 19:46:43.079793 18061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:43.081255 18061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:43.082471 18061 hybrid_clock.cc:648] HybridClock initialized: now 1786132003082427 us; error 38 us; skew 500 ppm
I20260807 19:46:43.082796 18061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:43 dist-test-slave-vq2q krb5kdc[18047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132003, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:43.086011 18061 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:46:43.087620 18061 webserver.cc:460] Webserver started at http://127.30.208.62:42495/ using document root <none> and password file <none>
I20260807 19:46:43.088303 18061 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:43.088410 18061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:43.088729 18061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:43.091831 18061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "106a32b7ddde4726a005cda4904c8ecb"
format_stamp: "Formatted at 2026-08-07 19:46:43 on dist-test-slave-vq2q"
server_key: "b34d5e47a5030b236a68aace4a53c94d"
server_key_iv: "75e55ca9c4f2bc38fb6386685233a52a"
server_key_version: "encryptionkey@0"
I20260807 19:46:43.092540 18061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "106a32b7ddde4726a005cda4904c8ecb"
format_stamp: "Formatted at 2026-08-07 19:46:43 on dist-test-slave-vq2q"
server_key: "b34d5e47a5030b236a68aace4a53c94d"
server_key_iv: "75e55ca9c4f2bc38fb6386685233a52a"
server_key_version: "encryptionkey@0"
I20260807 19:46:43.097795 18061 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:46:43.101377 18077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:43.102691 18061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:46:43.102837 18061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "106a32b7ddde4726a005cda4904c8ecb"
format_stamp: "Formatted at 2026-08-07 19:46:43 on dist-test-slave-vq2q"
server_key: "b34d5e47a5030b236a68aace4a53c94d"
server_key_iv: "75e55ca9c4f2bc38fb6386685233a52a"
server_key_version: "encryptionkey@0"
I20260807 19:46:43.103027 18061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:43.131752 18061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:43.135711 18061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:43.135962 18061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:43.153396 18061 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:34941
I20260807 19:46:43.153420 18133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:34941 every 8 connection(s)
I20260807 19:46:43.154763 18061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:46:43.158403 18134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:43.160290 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 18061
I20260807 19:46:43.160501 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:46:43.161126 31552 external_mini_cluster.cc:1428] Setting key 9967746d8f292109404280e46079e367
I20260807 19:46:43.170243 18134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb: Bootstrap starting.
I20260807 19:46:43.173903 18134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:43.175179 18134 log.cc:824] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:46:43 dist-test-slave-vq2q krb5kdc[18047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:43.178409 18134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb: No bootstrap required, opened a new log
I20260807 19:46:43.184340 18138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:43.163112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51244 (local address 127.30.208.62:34941)
0807 19:46:43.163945 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:43.163956 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:43.163998 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:46:43.165745 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:43.165768 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:43.166146 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:43.166469 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:43.168194 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:43.169502 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:43.171274 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:43.171935 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:43.172065 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:43.177070 (+ 5005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:43.177078 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:43.177093 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:43.177128 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:43.180266 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:43.180959 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:43.180966 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:43.180972 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:43.181149 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:43.181549 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:43.181554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:43.181556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:43.182094 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:43.182377 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:43.182836 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 19:46:43.183265 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":364,"threads_started":1}
I20260807 19:46:43.186847 18134 raft_consensus.cc:348] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106a32b7ddde4726a005cda4904c8ecb" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34941 } }
I20260807 19:46:43.187263 18134 raft_consensus.cc:374] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:43.187371 18134 raft_consensus.cc:729] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106a32b7ddde4726a005cda4904c8ecb, State: Initialized, Role: FOLLOWER
I20260807 19:46:43.187899 18134 consensus_queue.cc:260] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106a32b7ddde4726a005cda4904c8ecb" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34941 } }
I20260807 19:46:43.188119 18134 raft_consensus.cc:388] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:46:43.188215 18134 raft_consensus.cc:482] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:46:43.188333 18134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:43.190707 18134 raft_consensus.cc:504] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106a32b7ddde4726a005cda4904c8ecb" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34941 } }
I20260807 19:46:43.191160 18134 leader_election.cc:302] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 106a32b7ddde4726a005cda4904c8ecb; no voters:
I20260807 19:46:43.191671 18134 leader_election.cc:288] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:46:43.191807 18140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:46:43.192417 18140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [term 1 LEADER]: Becoming Leader. State: Replica: 106a32b7ddde4726a005cda4904c8ecb, State: Running, Role: LEADER
I20260807 19:46:43.192924 18140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106a32b7ddde4726a005cda4904c8ecb" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34941 } }
I20260807 19:46:43.193632 18134 sys_catalog.cc:564] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:46:43.195611 18142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 106a32b7ddde4726a005cda4904c8ecb. Latest consensus state: current_term: 1 leader_uuid: "106a32b7ddde4726a005cda4904c8ecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106a32b7ddde4726a005cda4904c8ecb" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34941 } } }
I20260807 19:46:43.196028 18142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:43.196043 18141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "106a32b7ddde4726a005cda4904c8ecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "106a32b7ddde4726a005cda4904c8ecb" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34941 } } }
I20260807 19:46:43.196664 18141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:43.198050 18145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:46:43.203383 18145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:46:43.212917 18145 catalog_manager.cc:1347] Generated new cluster ID: 79c0f1d14403441194da32872071e2f9
I20260807 19:46:43.213037 18145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:46:43.224938 18145 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:46:43.226259 18145 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:46:43.235077 18145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb: Generated new TSK 0
I20260807 19:46:43.236222 18145 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:46:43.313899 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34941
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:40083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:43.647763 18161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:43.648048 18161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:43.648177 18161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:43.655476 18161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:43.655592 18161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:43.655745 18161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:46:43.665544 18161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:40083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:43.666875 18161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:43.668187 18161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:43.680696 18168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:43.680779 18170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:43.680742 18167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:43.682156 18161 server_base.cc:1034] running on GCE node
I20260807 19:46:43.682825 18161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:43.683845 18161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:43.685072 18161 hybrid_clock.cc:648] HybridClock initialized: now 1786132003685013 us; error 54 us; skew 500 ppm
I20260807 19:46:43.685407 18161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:43 dist-test-slave-vq2q krb5kdc[18047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132003, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:43.688583 18161 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:46:43.690197 18161 webserver.cc:460] Webserver started at http://127.30.208.1:46523/ using document root <none> and password file <none>
I20260807 19:46:43.690891 18161 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:43.691018 18161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:43.691324 18161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:43.694444 18161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "91a17b612ee8480e904a6e9e3ee58a8f"
format_stamp: "Formatted at 2026-08-07 19:46:43 on dist-test-slave-vq2q"
server_key: "c4001d7f5eeb40727c5c9e754ae85f24"
server_key_iv: "d687caeb2eb2b501dc93cbdc0dca7fc3"
server_key_version: "encryptionkey@0"
I20260807 19:46:43.695261 18161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "91a17b612ee8480e904a6e9e3ee58a8f"
format_stamp: "Formatted at 2026-08-07 19:46:43 on dist-test-slave-vq2q"
server_key: "c4001d7f5eeb40727c5c9e754ae85f24"
server_key_iv: "d687caeb2eb2b501dc93cbdc0dca7fc3"
server_key_version: "encryptionkey@0"
I20260807 19:46:43.700572 18161 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:46:43.704224 18177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:43.705734 18161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:46:43.705905 18161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "91a17b612ee8480e904a6e9e3ee58a8f"
format_stamp: "Formatted at 2026-08-07 19:46:43 on dist-test-slave-vq2q"
server_key: "c4001d7f5eeb40727c5c9e754ae85f24"
server_key_iv: "d687caeb2eb2b501dc93cbdc0dca7fc3"
server_key_version: "encryptionkey@0"
I20260807 19:46:43.706111 18161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:43.735769 18161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:43.740373 18161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:43.740664 18161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:43.741828 18161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:43.743261 18161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:43.743340 18161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:43.743455 18161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:43.743505 18161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:43.776188 18161 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:39155
I20260807 19:46:43.776233 18298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:39155 every 8 connection(s)
I20260807 19:46:43.777930 18161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:46:43.782841 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 18161
I20260807 19:46:43.783057 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:46:43.783815 31552 external_mini_cluster.cc:1428] Setting key ee2a375574c16a585676b45f60c2750e
Aug 07 19:46:43 dist-test-slave-vq2q krb5kdc[18047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132003, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:43.800695 18301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:43.780986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:48719 (local address 127.30.208.62:34941)
0807 19:46:43.781683 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:43.781689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:43.784809 (+ 3120us) server_negotiation.cc:407] Connection header received
0807 19:46:43.786839 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:43.786846 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:43.786930 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:43.787026 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:43.788550 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:43.789353 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:43.790863 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:43.791293 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:43.791416 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:43.795002 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:43.795008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:43.795012 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:43.795050 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:43.797652 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:43.798464 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:43.798469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:43.798473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:43.798587 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:43.799030 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:43.799033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:43.799035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:43.799291 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:43.799408 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:43.800248 (+ 840us) server_negotiation.cc:299] Negotiation successful
0807 19:46:43.800436 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":405,"threads_started":1}
I20260807 19:46:43.802016 18302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:43.781652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34941 (local address 127.30.208.1:48719)
0807 19:46:43.784661 (+ 3009us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:43.784709 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:43.786575 (+ 1866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:43.787277 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:43.787285 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:43.787816 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:43.788387 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:43.788404 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:43.789508 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:43.789513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:43.790722 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:43.790734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:43.791525 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:43.791532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:43.791750 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:43.792391 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:43.792412 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:43.794786 (+ 2374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:43.797782 (+ 2996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:43.797791 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:43.797807 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:43.798328 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:43.798678 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:43.798682 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:43.798685 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:43.798931 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:43.799411 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:43.799418 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:43.799852 (+ 434us) client_negotiation.cc:769] Sending connection context
0807 19:46:43.800170 (+ 318us) client_negotiation.cc:240] Negotiation successful
0807 19:46:43.800671 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2728,"thread_start_us":456,"threads_started":1}
I20260807 19:46:43.803575 18299 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34941
I20260807 19:46:43.803872 18299 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:43.804561 18299 heartbeater.cc:511] Master 127.30.208.62:34941 requested a full tablet report, sending...
I20260807 19:46:43.806669 18096 ts_manager.cc:194] Registered new tserver with Master: 91a17b612ee8480e904a6e9e3ee58a8f (127.30.208.1:39155)
I20260807 19:46:43.808159 18096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:48719
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:46:43.848917 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34941
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:40083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:44.188352 18305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:44.188678 18305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:44.188814 18305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:44.196211 18305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:44.196343 18305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:44.196489 18305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:46:44.206681 18305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:40083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:44.208045 18305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:44.209438 18305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:44.222481 18310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:44.222519 18312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:44.222548 18314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:44.223381 18305 server_base.cc:1034] running on GCE node
I20260807 19:46:44.223950 18305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:44.224866 18305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:44.226055 18305 hybrid_clock.cc:648] HybridClock initialized: now 1786132004225998 us; error 50 us; skew 500 ppm
I20260807 19:46:44.226382 18305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:44 dist-test-slave-vq2q krb5kdc[18047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132004, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:44.229405 18305 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:46:44.230789 18305 webserver.cc:460] Webserver started at http://127.30.208.2:36293/ using document root <none> and password file <none>
I20260807 19:46:44.231467 18305 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:44.231586 18305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:44.231889 18305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:44.234863 18305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "0985ead54472445f986af8abfb4c08bb"
format_stamp: "Formatted at 2026-08-07 19:46:44 on dist-test-slave-vq2q"
server_key: "6ceb09a88486ef229476effcb90c9179"
server_key_iv: "64160b9d4197b4fd91296b4e3a73b580"
server_key_version: "encryptionkey@0"
I20260807 19:46:44.235761 18305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "0985ead54472445f986af8abfb4c08bb"
format_stamp: "Formatted at 2026-08-07 19:46:44 on dist-test-slave-vq2q"
server_key: "6ceb09a88486ef229476effcb90c9179"
server_key_iv: "64160b9d4197b4fd91296b4e3a73b580"
server_key_version: "encryptionkey@0"
I20260807 19:46:44.241096 18305 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:46:44.244568 18321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:44.245980 18305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 19:46:44.246127 18305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "0985ead54472445f986af8abfb4c08bb"
format_stamp: "Formatted at 2026-08-07 19:46:44 on dist-test-slave-vq2q"
server_key: "6ceb09a88486ef229476effcb90c9179"
server_key_iv: "64160b9d4197b4fd91296b4e3a73b580"
server_key_version: "encryptionkey@0"
I20260807 19:46:44.246313 18305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:44.288208 18305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:44.292523 18305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:44.292811 18305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:44.293889 18305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:44.295368 18305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:44.295444 18305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:44.295558 18305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:44.295607 18305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:44.326851 18305 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:39251
I20260807 19:46:44.326927 18439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:39251 every 8 connection(s)
I20260807 19:46:44.328155 18305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:46:44.328436 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 18305
I20260807 19:46:44.328619 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:46:44.329262 31552 external_mini_cluster.cc:1428] Setting key 46c12382aeacc508be5cc5d69326bb53
Aug 07 19:46:44 dist-test-slave-vq2q krb5kdc[18047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132004, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:44.348438 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:44.330877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:36575 (local address 127.30.208.62:34941)
0807 19:46:44.331314 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:44.331320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:44.332416 (+ 1096us) server_negotiation.cc:407] Connection header received
0807 19:46:44.334265 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:44.334271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:44.334347 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:44.334435 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:44.335922 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.336635 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.338183 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.338587 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.338694 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:44.342256 (+ 3562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:44.342262 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:44.342265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:44.342302 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:44.345505 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:44.346222 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:44.346227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:44.346231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:44.346379 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:44.346748 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:44.346752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:44.346755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:44.347089 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:44.347243 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:44.347878 (+ 635us) server_negotiation.cc:299] Negotiation successful
0807 19:46:44.348090 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":215,"threads_started":1}
I20260807 19:46:44.349308 18443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:44.331506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34941 (local address 127.30.208.2:36575)
0807 19:46:44.332281 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:44.332319 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:44.334015 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:44.334659 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:44.334667 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:44.335172 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:44.335753 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:44.335770 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.336782 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.336786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:44.338041 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:44.338053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.338802 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.338810 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:44.339019 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:44.339644 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:44.339666 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:44.342013 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:44.345631 (+ 3618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:44.345637 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:44.345650 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:44.346089 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:44.346464 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:44.346467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:44.346470 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:44.346641 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:44.347200 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:44.347205 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:44.347519 (+ 314us) client_negotiation.cc:769] Sending connection context
0807 19:46:44.347739 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 19:46:44.348128 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":327,"threads_started":1}
I20260807 19:46:44.350770 18440 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34941
I20260807 19:46:44.351119 18440 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:44.351948 18440 heartbeater.cc:511] Master 127.30.208.62:34941 requested a full tablet report, sending...
I20260807 19:46:44.353524 18096 ts_manager.cc:194] Registered new tserver with Master: 0985ead54472445f986af8abfb4c08bb (127.30.208.2:39251)
I20260807 19:46:44.354343 18096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:36575
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:46:44.396100 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34941
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:40083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:44.732728 18446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:44.733070 18446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:44.733247 18446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:44.740694 18446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:44.740866 18446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:44.741027 18446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:46:44.751291 18446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:40083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:44.752604 18446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:44.753947 18446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:44.767352 18451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:44.767513 18452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:44.767344 18454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:44.768124 18446 server_base.cc:1034] running on GCE node
I20260807 19:46:44.768800 18446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:44.769862 18446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:44.771070 18446 hybrid_clock.cc:648] HybridClock initialized: now 1786132004771014 us; error 47 us; skew 500 ppm
I20260807 19:46:44.771392 18446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:44 dist-test-slave-vq2q krb5kdc[18047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132004, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:44.774621 18446 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:46:44.776384 18446 webserver.cc:460] Webserver started at http://127.30.208.3:34007/ using document root <none> and password file <none>
I20260807 19:46:44.777340 18446 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:44.777477 18446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:44.777850 18446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:44.781991 18446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "f90d992151794f408008f2a00b06e8f6"
format_stamp: "Formatted at 2026-08-07 19:46:44 on dist-test-slave-vq2q"
server_key: "495cf0931267915b35406fd3d39e4216"
server_key_iv: "eb8129c4042befea0b168aeb80e62e00"
server_key_version: "encryptionkey@0"
I20260807 19:46:44.782912 18446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "f90d992151794f408008f2a00b06e8f6"
format_stamp: "Formatted at 2026-08-07 19:46:44 on dist-test-slave-vq2q"
server_key: "495cf0931267915b35406fd3d39e4216"
server_key_iv: "eb8129c4042befea0b168aeb80e62e00"
server_key_version: "encryptionkey@0"
I20260807 19:46:44.788244 18446 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260807 19:46:44.791628 18461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:44.792886 18446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:46:44.793040 18446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "f90d992151794f408008f2a00b06e8f6"
format_stamp: "Formatted at 2026-08-07 19:46:44 on dist-test-slave-vq2q"
server_key: "495cf0931267915b35406fd3d39e4216"
server_key_iv: "eb8129c4042befea0b168aeb80e62e00"
server_key_version: "encryptionkey@0"
I20260807 19:46:44.793268 18446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:44.805750 18446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:44.809700 18446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:44.809989 18446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:44.810912 18446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:44.811173 18299 heartbeater.cc:503] Master 127.30.208.62:34941 was elected leader, sending a full tablet report...
I20260807 19:46:44.812527 18446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:44.812629 18446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:44.812767 18446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:44.812849 18446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:44.843564 18446 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:32871
I20260807 19:46:44.843616 18582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:32871 every 8 connection(s)
I20260807 19:46:44.844923 18446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:46:44.855000 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 18446
I20260807 19:46:44.855192 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:46:44.855772 31552 external_mini_cluster.cc:1428] Setting key 6376dab9384dbb711f6a45f9f9b4683c
Aug 07 19:46:44 dist-test-slave-vq2q krb5kdc[18047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132004, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:44.865115 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:44.847462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:35385 (local address 127.30.208.62:34941)
0807 19:46:44.847669 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:44.847675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:44.848832 (+ 1157us) server_negotiation.cc:407] Connection header received
0807 19:46:44.850971 (+ 2139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:44.850976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:44.851065 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:44.851148 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:44.852585 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.853387 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.855229 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.855816 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.855961 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:44.859673 (+ 3712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:44.859679 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:44.859682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:44.859715 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:44.862146 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:44.862920 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:44.862924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:44.862927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:44.863044 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:44.863454 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:44.863459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:44.863462 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:44.863733 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:44.863887 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:44.864558 (+ 671us) server_negotiation.cc:299] Negotiation successful
0807 19:46:44.864789 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:46:44.866093 18585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:44.847949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34941 (local address 127.30.208.3:35385)
0807 19:46:44.848664 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:44.848702 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:44.850720 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:44.851358 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:44.851366 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:44.851881 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:44.852446 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:44.852465 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.853554 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.853559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:44.855052 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:44.855066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.856033 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.856038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:44.856233 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:44.856960 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:44.856983 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:44.859479 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:44.862276 (+ 2797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:44.862286 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:44.862302 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:44.862807 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:44.863146 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:44.863151 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:44.863155 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:44.863351 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:44.863861 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:44.863870 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:44.864197 (+ 327us) client_negotiation.cc:769] Sending connection context
0807 19:46:44.864443 (+ 246us) client_negotiation.cc:240] Negotiation successful
0807 19:46:44.864963 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":287,"threads_started":1}
I20260807 19:46:44.867439 18583 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34941
I20260807 19:46:44.867748 18583 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:44.868467 18583 heartbeater.cc:511] Master 127.30.208.62:34941 requested a full tablet report, sending...
I20260807 19:46:44.869995 18096 ts_manager.cc:194] Registered new tserver with Master: f90d992151794f408008f2a00b06e8f6 (127.30.208.3:32871)
I20260807 19:46:44.870814 18096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:35385
I20260807 19:46:44.871492 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:46:44.889958 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:44.877035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51250 (local address 127.30.208.62:34941)
0807 19:46:44.877280 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:44.877288 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:44.877692 (+ 404us) server_negotiation.cc:407] Connection header received
0807 19:46:44.877837 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:44.877841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:44.877924 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:44.878007 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:44.879452 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.880203 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.881617 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.882037 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.882134 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:44.884600 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:44.884605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:44.884608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:44.884640 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:44.887167 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:44.888044 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:44.888048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:44.888051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:44.888195 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:44.888499 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:44.888502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:44.888504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:44.888813 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:44.889020 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:44.889396 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 19:46:44.889585 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:46:44.894176 18096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:46:44.901386 18096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:46:44.939505 18599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:44.913969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39155 (local address 127.0.0.1:60748)
0807 19:46:44.929830 (+ 15861us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:44.929852 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:44.930029 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:44.931085 (+ 1056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:44.931095 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:44.931131 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:44.931614 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:44.931628 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.934222 (+ 2594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.934227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:44.936485 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:44.936501 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.938452 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.938458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:44.938652 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:44.938708 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 19:46:44.938854 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 19:46:44.939052 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15717,"mutex_wait_us":45,"thread_start_us":18209,"threads_started":1}
I20260807 19:46:44.939906 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:44.913385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.30.208.2:39251)
0807 19:46:44.930538 (+ 17153us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:44.930546 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:44.930572 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:46:44.930660 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:44.930669 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:44.930890 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:44.931997 (+ 1107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:44.932738 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.934260 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.937279 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.938630 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.938777 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:44.938963 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:44.939224 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 19:46:44.939407 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17001,"thread_start_us":16888,"threads_started":1}
I20260807 19:46:44.940232 18600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:44.914868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.30.208.1:39155)
0807 19:46:44.929414 (+ 14546us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:44.929426 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:44.929976 (+ 550us) server_negotiation.cc:407] Connection header received
0807 19:46:44.930247 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:44.930258 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:44.930555 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:44.930790 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:44.931837 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.933993 (+ 2156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.936842 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.938246 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.938387 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:44.938633 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:44.939762 (+ 1129us) server_negotiation.cc:299] Negotiation successful
0807 19:46:44.939930 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14325,"thread_start_us":14132,"threads_started":1}
I20260807 19:46:44.939805 18595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:44.913229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:39251 (local address 127.0.0.1:35958)
0807 19:46:44.914064 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:44.914100 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:44.914270 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:44.931992 (+ 17722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:44.931996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:44.932022 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:44.932534 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:44.932548 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.934389 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.934394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:44.936462 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:44.936477 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.938856 (+ 2379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.938863 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:44.939042 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:44.939075 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:46:44.939172 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 19:46:44.939307 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":109,"thread_start_us":406,"threads_started":1}
I20260807 19:46:44.940747 18597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:44.914438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:32871 (local address 127.0.0.1:35912)
0807 19:46:44.932523 (+ 18085us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:44.932542 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:44.932658 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:44.933717 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:44.933722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:44.933760 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:44.934076 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:44.934086 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.936024 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.936031 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:44.937992 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:44.938009 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.939996 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.940003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:44.940170 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:44.940202 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:46:44.940272 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 19:46:44.940413 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17944,"thread_start_us":15100,"threads_started":1}
I20260807 19:46:44.940863 18601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:44.914866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35912 (local address 127.30.208.3:32871)
0807 19:46:44.932893 (+ 18027us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:44.932901 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:44.932934 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:46:44.933027 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:44.933034 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:44.933269 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:44.933451 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:44.934307 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.935856 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.938184 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:44.939538 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:44.939657 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:44.939868 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:44.940360 (+ 492us) server_negotiation.cc:299] Negotiation successful
0807 19:46:44.940524 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17820,"thread_start_us":17849,"threads_started":1}
I20260807 19:46:44.944665 18514 tablet_service.cc:1467] Processing CreateTablet for tablet 357ccdbdcbeb4137baeea1ebc0b1c45c (DEFAULT_TABLE table=test-table [id=3b2b9cb4e3ac44a092cff1d729a9a730]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:44.944537 18228 tablet_service.cc:1467] Processing CreateTablet for tablet 357ccdbdcbeb4137baeea1ebc0b1c45c (DEFAULT_TABLE table=test-table [id=3b2b9cb4e3ac44a092cff1d729a9a730]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:44.944541 18370 tablet_service.cc:1467] Processing CreateTablet for tablet 357ccdbdcbeb4137baeea1ebc0b1c45c (DEFAULT_TABLE table=test-table [id=3b2b9cb4e3ac44a092cff1d729a9a730]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:44.945906 18514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 357ccdbdcbeb4137baeea1ebc0b1c45c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:44.945971 18370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 357ccdbdcbeb4137baeea1ebc0b1c45c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:44.945950 18228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 357ccdbdcbeb4137baeea1ebc0b1c45c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:44.957158 18602 tablet_bootstrap.cc:492] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6: Bootstrap starting.
I20260807 19:46:44.960273 18604 tablet_bootstrap.cc:492] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f: Bootstrap starting.
I20260807 19:46:44.960850 18603 tablet_bootstrap.cc:492] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb: Bootstrap starting.
I20260807 19:46:44.961911 18602 tablet_bootstrap.cc:654] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:44.963187 18602 log.cc:824] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:44.964213 18603 tablet_bootstrap.cc:654] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:44.965423 18603 log.cc:824] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:44.965876 18602 tablet_bootstrap.cc:492] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6: No bootstrap required, opened a new log
I20260807 19:46:44.966204 18602 ts_tablet_manager.cc:1397] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260807 19:46:44.967280 18604 tablet_bootstrap.cc:654] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:44.968560 18604 log.cc:824] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:44.972301 18603 tablet_bootstrap.cc:492] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb: No bootstrap required, opened a new log
I20260807 19:46:44.972599 18603 ts_tablet_manager.cc:1397] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260807 19:46:44.973085 18604 tablet_bootstrap.cc:492] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f: No bootstrap required, opened a new log
I20260807 19:46:44.973373 18604 ts_tablet_manager.cc:1397] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.007s
I20260807 19:46:44.974530 18602 raft_consensus.cc:348] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0985ead54472445f986af8abfb4c08bb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39251 } } peers { permanent_uuid: "f90d992151794f408008f2a00b06e8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 32871 } } peers { permanent_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39155 } }
I20260807 19:46:44.974898 18602 raft_consensus.cc:374] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:44.975018 18602 raft_consensus.cc:729] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f90d992151794f408008f2a00b06e8f6, State: Initialized, Role: FOLLOWER
I20260807 19:46:44.975594 18602 consensus_queue.cc:260] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0985ead54472445f986af8abfb4c08bb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39251 } } peers { permanent_uuid: "f90d992151794f408008f2a00b06e8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 32871 } } peers { permanent_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39155 } }
I20260807 19:46:44.977878 18583 heartbeater.cc:503] Master 127.30.208.62:34941 was elected leader, sending a full tablet report...
I20260807 19:46:44.978546 18602 ts_tablet_manager.cc:1428] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260807 19:46:44.983659 18603 raft_consensus.cc:348] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0985ead54472445f986af8abfb4c08bb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39251 } } peers { permanent_uuid: "f90d992151794f408008f2a00b06e8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 32871 } } peers { permanent_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39155 } }
I20260807 19:46:44.983659 18604 raft_consensus.cc:348] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0985ead54472445f986af8abfb4c08bb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39251 } } peers { permanent_uuid: "f90d992151794f408008f2a00b06e8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 32871 } } peers { permanent_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39155 } }
I20260807 19:46:44.984139 18603 raft_consensus.cc:374] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:44.984138 18604 raft_consensus.cc:374] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:44.984244 18604 raft_consensus.cc:729] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91a17b612ee8480e904a6e9e3ee58a8f, State: Initialized, Role: FOLLOWER
I20260807 19:46:44.984246 18603 raft_consensus.cc:729] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0985ead54472445f986af8abfb4c08bb, State: Initialized, Role: FOLLOWER
I20260807 19:46:44.984894 18604 consensus_queue.cc:260] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0985ead54472445f986af8abfb4c08bb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39251 } } peers { permanent_uuid: "f90d992151794f408008f2a00b06e8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 32871 } } peers { permanent_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39155 } }
I20260807 19:46:44.984895 18603 consensus_queue.cc:260] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0985ead54472445f986af8abfb4c08bb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39251 } } peers { permanent_uuid: "f90d992151794f408008f2a00b06e8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 32871 } } peers { permanent_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39155 } }
I20260807 19:46:44.986544 18440 heartbeater.cc:503] Master 127.30.208.62:34941 was elected leader, sending a full tablet report...
I20260807 19:46:44.986908 18604 ts_tablet_manager.cc:1428] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260807 19:46:44.987947 18603 ts_tablet_manager.cc:1428] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260807 19:46:45.022388 18609 raft_consensus.cc:482] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:45.022692 18609 raft_consensus.cc:504] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0985ead54472445f986af8abfb4c08bb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39251 } } peers { permanent_uuid: "f90d992151794f408008f2a00b06e8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 32871 } } peers { permanent_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39155 } }
I20260807 19:46:45.024430 18609 leader_election.cc:288] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0985ead54472445f986af8abfb4c08bb (127.30.208.2:39251), f90d992151794f408008f2a00b06e8f6 (127.30.208.3:32871)
W20260807 19:46:45.030670 18300 tablet.cc:2369] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:46:45.032351 18611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.024904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:39251 (local address 127.30.208.1:38669)
0807 19:46:45.025696 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:45.025716 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:45.025867 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:45.026271 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:45.026276 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:45.026304 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:45.026651 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:45.026661 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.027621 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.027624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:45.029496 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:45.029516 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.031506 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.031513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:45.031697 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:45.031733 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:46:45.031833 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 19:46:45.031972 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":1,"thread_start_us":446,"threads_started":1}
I20260807 19:46:45.032393 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.024952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:38669 (local address 127.30.208.2:39251)
0807 19:46:45.025139 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:45.025143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:45.025802 (+ 659us) server_negotiation.cc:407] Connection header received
0807 19:46:45.026045 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:45.026051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:45.026129 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:45.026220 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:45.026785 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.027499 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.029690 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.031055 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.031247 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:45.031322 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:45.031928 (+ 606us) server_negotiation.cc:299] Negotiation successful
0807 19:46:45.032127 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:46:45.033629 18390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "357ccdbdcbeb4137baeea1ebc0b1c45c" candidate_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0985ead54472445f986af8abfb4c08bb" is_pre_election: true
I20260807 19:46:45.034103 18601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.026072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:46619 (local address 127.30.208.3:32871)
0807 19:46:45.026290 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:45.026296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:45.026779 (+ 483us) server_negotiation.cc:407] Connection header received
0807 19:46:45.026937 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:45.026941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:45.026998 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:45.027069 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:45.027581 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.028445 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.032152 (+ 3707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.033201 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.033373 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:45.033442 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:45.033717 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 19:46:45.033858 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:46:45.034251 18613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.025822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:32871 (local address 127.30.208.1:46619)
0807 19:46:45.026679 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:45.026693 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:45.026806 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:45.027091 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:45.027095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:45.027116 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:45.027439 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:45.027447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.028595 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.028600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:45.031950 (+ 3350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:45.031969 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.033409 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.033414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:45.033552 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:45.033577 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:46:45.033666 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 19:46:45.033795 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":542,"threads_started":1}
I20260807 19:46:45.034193 18608 raft_consensus.cc:482] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:45.034235 18390 raft_consensus.cc:2454] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91a17b612ee8480e904a6e9e3ee58a8f in term 0.
I20260807 19:46:45.034477 18608 raft_consensus.cc:504] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0985ead54472445f986af8abfb4c08bb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39251 } } peers { permanent_uuid: "f90d992151794f408008f2a00b06e8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 32871 } } peers { permanent_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39155 } }
I20260807 19:46:45.035000 18181 leader_election.cc:302] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0985ead54472445f986af8abfb4c08bb, 91a17b612ee8480e904a6e9e3ee58a8f; no voters:
I20260807 19:46:45.035497 18609 raft_consensus.cc:2781] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:46:45.035351 18534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "357ccdbdcbeb4137baeea1ebc0b1c45c" candidate_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f90d992151794f408008f2a00b06e8f6" is_pre_election: true
I20260807 19:46:45.035643 18609 raft_consensus.cc:482] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:46:45.035745 18609 raft_consensus.cc:3037] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:45.035872 18534 raft_consensus.cc:2454] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91a17b612ee8480e904a6e9e3ee58a8f in term 0.
I20260807 19:46:45.036499 18608 leader_election.cc:288] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0985ead54472445f986af8abfb4c08bb (127.30.208.2:39251), 91a17b612ee8480e904a6e9e3ee58a8f (127.30.208.1:39155)
I20260807 19:46:45.039112 18609 raft_consensus.cc:504] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0985ead54472445f986af8abfb4c08bb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39251 } } peers { permanent_uuid: "f90d992151794f408008f2a00b06e8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 32871 } } peers { permanent_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39155 } }
I20260807 19:46:45.039785 18609 leader_election.cc:288] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [CANDIDATE]: Term 1 election: Requested vote from peers 0985ead54472445f986af8abfb4c08bb (127.30.208.2:39251), f90d992151794f408008f2a00b06e8f6 (127.30.208.3:32871)
I20260807 19:46:45.040306 18390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "357ccdbdcbeb4137baeea1ebc0b1c45c" candidate_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0985ead54472445f986af8abfb4c08bb"
I20260807 19:46:45.040545 18390 raft_consensus.cc:3037] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:45.040755 18534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "357ccdbdcbeb4137baeea1ebc0b1c45c" candidate_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f90d992151794f408008f2a00b06e8f6"
I20260807 19:46:45.040984 18534 raft_consensus.cc:3037] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:45.043879 18585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.036832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:39251 (local address 127.30.208.3:52735)
0807 19:46:45.037028 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:45.037047 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:45.037160 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:45.037902 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:45.037906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:45.037937 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:45.038128 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:45.038137 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.039165 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.039170 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:45.041279 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:45.041294 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.043094 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.043099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:45.043320 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:45.043355 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:46:45.043444 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 19:46:45.043581 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":97}
I20260807 19:46:45.044210 18390 raft_consensus.cc:2454] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91a17b612ee8480e904a6e9e3ee58a8f in term 1.
I20260807 19:46:45.044351 18534 raft_consensus.cc:2454] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91a17b612ee8480e904a6e9e3ee58a8f in term 1.
I20260807 19:46:45.044723 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.037305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:52735 (local address 127.30.208.2:39251)
0807 19:46:45.037475 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:45.037480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:45.037498 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:46:45.037556 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:45.037560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:45.037630 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:45.037728 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:45.038264 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.038993 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.041481 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.042774 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.042941 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:45.043012 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:45.044215 (+ 1203us) server_negotiation.cc:299] Negotiation successful
0807 19:46:45.044350 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 19:46:45.044662 18181 leader_election.cc:302] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0985ead54472445f986af8abfb4c08bb, 91a17b612ee8480e904a6e9e3ee58a8f; no voters:
I20260807 19:46:45.045269 18609 raft_consensus.cc:2781] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:46:45.045436 18600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.036961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:33417 (local address 127.30.208.1:39155)
0807 19:46:45.037142 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:45.037149 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:45.037944 (+ 795us) server_negotiation.cc:407] Connection header received
0807 19:46:45.038200 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:45.038205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:45.038275 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:45.038328 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:45.039090 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.040216 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.042994 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.044151 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.044275 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:45.044342 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:45.045012 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 19:46:45.045149 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 19:46:45.045480 18614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.037272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39155 (local address 127.30.208.3:33417)
0807 19:46:45.037850 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:45.037868 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:45.038057 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:45.038502 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:45.038506 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:45.038530 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:45.038901 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:45.038914 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.040374 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.040379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:45.042789 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:45.042808 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.044618 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.044625 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:45.044796 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:45.044828 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:46:45.044962 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 19:46:45.045093 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":22,"thread_start_us":569,"threads_started":1}
I20260807 19:46:45.045300 18389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "357ccdbdcbeb4137baeea1ebc0b1c45c" candidate_uuid: "f90d992151794f408008f2a00b06e8f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0985ead54472445f986af8abfb4c08bb" is_pre_election: true
I20260807 19:46:45.045727 18389 raft_consensus.cc:2379] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f90d992151794f408008f2a00b06e8f6 in current term 1: Already voted for candidate 91a17b612ee8480e904a6e9e3ee58a8f in this term.
I20260807 19:46:45.046617 18609 raft_consensus.cc:686] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [term 1 LEADER]: Becoming Leader. State: Replica: 91a17b612ee8480e904a6e9e3ee58a8f, State: Running, Role: LEADER
I20260807 19:46:45.047060 18249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "357ccdbdcbeb4137baeea1ebc0b1c45c" candidate_uuid: "f90d992151794f408008f2a00b06e8f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" is_pre_election: true
I20260807 19:46:45.047128 18609 consensus_queue.cc:237] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0985ead54472445f986af8abfb4c08bb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39251 } } peers { permanent_uuid: "f90d992151794f408008f2a00b06e8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 32871 } } peers { permanent_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39155 } }
I20260807 19:46:45.047966 18462 leader_election.cc:302] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f90d992151794f408008f2a00b06e8f6; no voters: 0985ead54472445f986af8abfb4c08bb, 91a17b612ee8480e904a6e9e3ee58a8f
I20260807 19:46:45.048391 18608 raft_consensus.cc:2726] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:46:45.052302 18096 catalog_manager.cc:5495] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 91a17b612ee8480e904a6e9e3ee58a8f (127.30.208.1). New cstate: current_term: 1 leader_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0985ead54472445f986af8abfb4c08bb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f90d992151794f408008f2a00b06e8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 32871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91a17b612ee8480e904a6e9e3ee58a8f" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39155 } health_report { overall_health: HEALTHY } } }
W20260807 19:46:45.080367 18441 tablet.cc:2369] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:46:45.096467 18600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.090838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60756 (local address 127.30.208.1:39155)
0807 19:46:45.091036 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:45.091042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:45.091143 (+ 101us) server_negotiation.cc:407] Connection header received
0807 19:46:45.091321 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:45.091325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:45.091407 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:45.091531 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:46:45.091958 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.092744 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.094157 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.094615 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.094757 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:45.095240 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:46:45.095800 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:46:45.095917 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:45.096069 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 19:46:45.096156 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 19:46:45.096539 18584 tablet.cc:2369] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:46:45.104903 18534 raft_consensus.cc:1261] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [term 1 FOLLOWER]: Refusing update from remote peer 91a17b612ee8480e904a6e9e3ee58a8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:46:45.104876 18389 raft_consensus.cc:1261] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb [term 1 FOLLOWER]: Refusing update from remote peer 91a17b612ee8480e904a6e9e3ee58a8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:46:45.105823 18615 consensus_queue.cc:1035] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0985ead54472445f986af8abfb4c08bb" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:46:45.106225 18609 consensus_queue.cc:1035] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f90d992151794f408008f2a00b06e8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 32871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:46:45.124723 18619 mvcc.cc:205] Tried to move back new op lower bound from 7315996692899545088 to 7315996692677066752. Current Snapshot: MvccSnapshot[applied={T|T < 7315996692899545088}]
I20260807 19:46:45.126493 18621 mvcc.cc:205] Tried to move back new op lower bound from 7315996692899545088 to 7315996692677066752. Current Snapshot: MvccSnapshot[applied={T|T < 7315996692899545088}]
I20260807 19:46:45.142032 18096 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:46:45.142290 18096 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:46:45.146001 18096 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 106a32b7ddde4726a005cda4904c8ecb: Sending DeleteTablet for 3 replicas of tablet 357ccdbdcbeb4137baeea1ebc0b1c45c
I20260807 19:46:45.147203 18370 tablet_service.cc:1514] Processing DeleteTablet for tablet 357ccdbdcbeb4137baeea1ebc0b1c45c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:45 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:35958
I20260807 19:46:45.147269 18228 tablet_service.cc:1514] Processing DeleteTablet for tablet 357ccdbdcbeb4137baeea1ebc0b1c45c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:45 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:60748
I20260807 19:46:45.147454 18514 tablet_service.cc:1514] Processing DeleteTablet for tablet 357ccdbdcbeb4137baeea1ebc0b1c45c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:45 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:35912
I20260807 19:46:45.148962 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 18161
Aug 07 19:46:45 dist-test-slave-vq2q krb5kdc[18047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:46:45.176400 18630 tablet_replica.cc:330] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f: stopping tablet replica
I20260807 19:46:45.176975 18630 raft_consensus.cc:2229] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:46:45.177857 18630 raft_consensus.cc:2258] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 91a17b612ee8480e904a6e9e3ee58a8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:45.178699 18632 tablet_replica.cc:330] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6: stopping tablet replica
I20260807 19:46:45.179060 18632 raft_consensus.cc:2229] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:46:45.179553 18632 raft_consensus.cc:2258] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:45.181499 18632 ts_tablet_manager.cc:1905] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:45.185299 18600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.152729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60770 (local address 127.30.208.1:39155)
0807 19:46:45.152960 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:45.152967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:45.158334 (+ 5367us) server_negotiation.cc:407] Connection header received
0807 19:46:45.159267 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:45.159273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:45.159373 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:45.159440 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:45.162862 (+ 3422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.164072 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.168005 (+ 3933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.168800 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.168908 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:45.177346 (+ 8438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:45.177362 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:45.177380 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:45.177435 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:45.181485 (+ 4050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:45.182453 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:45.182463 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:45.182467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:45.182635 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:45.183146 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:45.183151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:45.183154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:45.183685 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:45.184016 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:45.184650 (+ 634us) server_negotiation.cc:299] Negotiation successful
0807 19:46:45.184922 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 19:46:45.186869 18293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:45.187497 18631 tablet_replica.cc:330] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb: stopping tablet replica
I20260807 19:46:45.188100 18631 raft_consensus.cc:2229] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:46:45.188735 18631 raft_consensus.cc:2258] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:45.189468 18632 ts_tablet_manager.cc:1918] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:45.189636 18632 log.cc:1196] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/357ccdbdcbeb4137baeea1ebc0b1c45c
I20260807 19:46:45.190186 18632 ts_tablet_manager.cc:1939] T 357ccdbdcbeb4137baeea1ebc0b1c45c P f90d992151794f408008f2a00b06e8f6: Deleting consensus metadata
I20260807 19:46:45.191418 18631 ts_tablet_manager.cc:1905] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:45.192225 18081 catalog_manager.cc:4841] TS f90d992151794f408008f2a00b06e8f6 (127.30.208.3:32871): tablet 357ccdbdcbeb4137baeea1ebc0b1c45c (table test-table [id=3b2b9cb4e3ac44a092cff1d729a9a730]) successfully deleted
I20260807 19:46:45.198738 18631 ts_tablet_manager.cc:1918] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:45.198945 18631 log.cc:1196] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/357ccdbdcbeb4137baeea1ebc0b1c45c
I20260807 19:46:45.199522 18631 ts_tablet_manager.cc:1939] T 357ccdbdcbeb4137baeea1ebc0b1c45c P 0985ead54472445f986af8abfb4c08bb: Deleting consensus metadata
I20260807 19:46:45.201529 18081 catalog_manager.cc:4841] TS 0985ead54472445f986af8abfb4c08bb (127.30.208.2:39251): tablet 357ccdbdcbeb4137baeea1ebc0b1c45c (table test-table [id=3b2b9cb4e3ac44a092cff1d729a9a730]) successfully deleted
I20260807 19:46:45.289589 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 18161
I20260807 19:46:45.308547 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 18305
W20260807 19:46:45.308645 18078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.208.1:39155 (error 108)
I20260807 19:46:45.310503 18599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.310062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39155 (local address 127.0.0.1:60772)
0807 19:46:45.310244 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:45.310368 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 19:46:45.310714 18078 catalog_manager.cc:4584] TS 91a17b612ee8480e904a6e9e3ee58a8f (127.30.208.1:39155): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 357ccdbdcbeb4137baeea1ebc0b1c45c: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39155: connect: Connection refused (error 111)
Aug 07 19:46:45 dist-test-slave-vq2q krb5kdc[18047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:46:45.324029 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.309316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35966 (local address 127.30.208.2:39251)
0807 19:46:45.309500 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:45.309506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:45.309804 (+ 298us) server_negotiation.cc:407] Connection header received
0807 19:46:45.309973 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:45.309978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:45.310053 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:45.310142 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:45.311972 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.312709 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.313975 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.314385 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.314480 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:45.318555 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:45.318569 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:45.318584 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:45.318622 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:45.321604 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:45.322248 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:45.322255 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:45.322257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:45.322382 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:45.322718 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:45.322721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:45.322723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:45.323076 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:45.323317 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:45.323541 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 19:46:45.323707 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:46:45.324608 18435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:45.364244 18599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.363803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39155 (local address 127.0.0.1:60774)
0807 19:46:45.363995 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:45.364081 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:46:45.365330 18599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.364942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39155 (local address 127.0.0.1:60784)
0807 19:46:45.365088 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:45.365170 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 19:46:45.414992 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 18305
I20260807 19:46:45.426075 18599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.425561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39155 (local address 127.0.0.1:60786)
0807 19:46:45.425792 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:45.425892 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 19:46:45.427096 18599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.426783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39155 (local address 127.0.0.1:60792)
0807 19:46:45.426912 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:45.426983 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260807 19:46:45.432898 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 18446
Aug 07 19:46:45 dist-test-slave-vq2q krb5kdc[18047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:46:45.447515 18601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.433592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35918 (local address 127.30.208.3:32871)
0807 19:46:45.433753 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:45.433757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:45.433776 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:46:45.433909 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:45.433913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:45.433976 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:45.434055 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:45.435303 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.436058 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.437505 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:45.437900 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:45.437992 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:45.442430 (+ 4438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:45.442445 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:45.442462 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:45.442498 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:45.445240 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:45.445868 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:45.445874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:45.445877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:45.446009 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:45.446289 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:45.446292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:45.446294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:45.446647 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:45.446854 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:45.447087 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 19:46:45.447244 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 19:46:45.448096 18578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:45.507798 18599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.507204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39155 (local address 127.0.0.1:60806)
0807 19:46:45.507442 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:45.507562 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 19:46:45.508975 18599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:45.508526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39155 (local address 127.0.0.1:60822)
0807 19:46:45.508742 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:45.508841 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260807 19:46:45.539417 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 18446
I20260807 19:46:45.556385 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 18061
I20260807 19:46:45.557592 18129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:45.668313 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 18061
2026-08-07T19:46:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4352 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T19:46:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:46:45Z Disabled control of system clock
I20260807 19:46:45.755079 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:41071
--webserver_interface=127.30.208.62
--webserver_port=0
--builtin_ntp_servers=127.30.208.20:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:41071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:46:46.096107 18642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:46.096417 18642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:46.096526 18642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:46.103879 18642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:46:46.103999 18642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:46.104045 18642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:46.104130 18642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:46:46.104187 18642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:46:46.113492 18642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:41071
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:41071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:46.114802 18642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:46.116596 18642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:46.128931 18648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:46.129379 18651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:46.128913 18647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:46.129560 18642 server_base.cc:1034] running on GCE node
I20260807 19:46:46.130451 18642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:46.131932 18642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:46.133141 18642 hybrid_clock.cc:648] HybridClock initialized: now 1786132006133083 us; error 52 us; skew 500 ppm
I20260807 19:46:46.133523 18642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:46:46.138530 18642 webserver.cc:460] Webserver started at http://127.30.208.62:45389/ using document root <none> and password file <none>
I20260807 19:46:46.139240 18642 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:46.139346 18642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:46.139667 18642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:46.142683 18642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "d71e9e35baef4de3b57dbc68d3c1f28b"
format_stamp: "Formatted at 2026-08-07 19:46:46 on dist-test-slave-vq2q"
server_key: "508c8a69e5c0705031b2f554fd694a76"
server_key_iv: "851f136891640dce2011b9100c4c3dfc"
server_key_version: "encryptionkey@0"
I20260807 19:46:46.143476 18642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "d71e9e35baef4de3b57dbc68d3c1f28b"
format_stamp: "Formatted at 2026-08-07 19:46:46 on dist-test-slave-vq2q"
server_key: "508c8a69e5c0705031b2f554fd694a76"
server_key_iv: "851f136891640dce2011b9100c4c3dfc"
server_key_version: "encryptionkey@0"
I20260807 19:46:46.148408 18642 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:46:46.151734 18657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:46.153003 18642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:46:46.153198 18642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "d71e9e35baef4de3b57dbc68d3c1f28b"
format_stamp: "Formatted at 2026-08-07 19:46:46 on dist-test-slave-vq2q"
server_key: "508c8a69e5c0705031b2f554fd694a76"
server_key_iv: "851f136891640dce2011b9100c4c3dfc"
server_key_version: "encryptionkey@0"
I20260807 19:46:46.153406 18642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:46.171139 18642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:46.171967 18642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:46.172227 18642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:46.189909 18642 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:41071
I20260807 19:46:46.189921 18710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:41071 every 8 connection(s)
I20260807 19:46:46.191255 18642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:46:46.193125 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 18642
I20260807 19:46:46.193320 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:46:46.193753 31552 external_mini_cluster.cc:1428] Setting key 7aa6a043cfea5a7a1b98df7ed743605c
I20260807 19:46:46.195353 18711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:46.208994 18714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:46.195511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35946 (local address 127.30.208.62:41071)
0807 19:46:46.196540 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:46.196562 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:46.196615 (+ 53us) server_negotiation.cc:407] Connection header received
0807 19:46:46.198565 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:46.198592 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:46.199193 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:46.199636 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:46.200182 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:46.202306 (+ 2124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:46.203796 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:46.204788 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:46.204981 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:46.206097 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:46.206108 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:46.206127 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:46.206901 (+ 774us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:46.207003 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:46.207030 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:46.207150 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:46.207387 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 19:46:46.207733 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":339,"threads_started":1}
I20260807 19:46:46.209910 18711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b: Bootstrap starting.
I20260807 19:46:46.212913 18711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:46.213925 18711 log.cc:824] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:46.216836 18711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b: No bootstrap required, opened a new log
I20260807 19:46:46.224128 18711 raft_consensus.cc:348] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71e9e35baef4de3b57dbc68d3c1f28b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41071 } }
I20260807 19:46:46.224519 18711 raft_consensus.cc:374] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:46.224622 18711 raft_consensus.cc:729] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d71e9e35baef4de3b57dbc68d3c1f28b, State: Initialized, Role: FOLLOWER
I20260807 19:46:46.225162 18711 consensus_queue.cc:260] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71e9e35baef4de3b57dbc68d3c1f28b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41071 } }
I20260807 19:46:46.225430 18711 raft_consensus.cc:388] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:46:46.225561 18711 raft_consensus.cc:482] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:46:46.225746 18711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:46.227896 18711 raft_consensus.cc:504] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71e9e35baef4de3b57dbc68d3c1f28b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41071 } }
I20260807 19:46:46.228422 18711 leader_election.cc:302] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d71e9e35baef4de3b57dbc68d3c1f28b; no voters:
I20260807 19:46:46.229048 18711 leader_election.cc:288] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:46:46.229244 18717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:46:46.229972 18717 raft_consensus.cc:686] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [term 1 LEADER]: Becoming Leader. State: Replica: d71e9e35baef4de3b57dbc68d3c1f28b, State: Running, Role: LEADER
I20260807 19:46:46.230494 18717 consensus_queue.cc:237] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71e9e35baef4de3b57dbc68d3c1f28b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41071 } }
I20260807 19:46:46.231155 18711 sys_catalog.cc:564] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:46:46.233379 18721 sys_catalog.cc:455] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d71e9e35baef4de3b57dbc68d3c1f28b. Latest consensus state: current_term: 1 leader_uuid: "d71e9e35baef4de3b57dbc68d3c1f28b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71e9e35baef4de3b57dbc68d3c1f28b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41071 } } }
I20260807 19:46:46.233858 18721 sys_catalog.cc:458] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:46.233659 18718 sys_catalog.cc:455] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d71e9e35baef4de3b57dbc68d3c1f28b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d71e9e35baef4de3b57dbc68d3c1f28b" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41071 } } }
I20260807 19:46:46.234155 18718 sys_catalog.cc:458] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:46.234879 18728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:46:46.240427 18728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:46:46.249991 18728 catalog_manager.cc:1347] Generated new cluster ID: a61ca78aa17c4b4c89ea739e198c327c
I20260807 19:46:46.250118 18728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:46:46.259150 18728 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:46:46.261026 18728 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:46:46.273774 18728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b: Generated new TSK 0
I20260807 19:46:46.274943 18728 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:46:46.290671 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41071
--builtin_ntp_servers=127.30.208.20:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:46:46.623906 18738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:46.624195 18738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:46.624289 18738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:46.631747 18738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:46.631858 18738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:46.631971 18738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:46:46.641273 18738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:46.642549 18738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:46.644326 18738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:46.657075 18744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:46.657168 18743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:46.657128 18746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:46.658346 18738 server_base.cc:1034] running on GCE node
I20260807 19:46:46.658967 18738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:46.659935 18738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:46.661139 18738 hybrid_clock.cc:648] HybridClock initialized: now 1786132006661095 us; error 33 us; skew 500 ppm
I20260807 19:46:46.661482 18738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:46:46.663668 18738 webserver.cc:460] Webserver started at http://127.30.208.1:37615/ using document root <none> and password file <none>
I20260807 19:46:46.664320 18738 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:46.664435 18738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:46.664732 18738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:46.667766 18738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "f8d464853a31407782fca51ad788727c"
format_stamp: "Formatted at 2026-08-07 19:46:46 on dist-test-slave-vq2q"
server_key: "4e69efb6a587948ed203bae606cbd1d5"
server_key_iv: "17b728e00b62d9b3fe5727d4d5a78fbc"
server_key_version: "encryptionkey@0"
I20260807 19:46:46.668526 18738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "f8d464853a31407782fca51ad788727c"
format_stamp: "Formatted at 2026-08-07 19:46:46 on dist-test-slave-vq2q"
server_key: "4e69efb6a587948ed203bae606cbd1d5"
server_key_iv: "17b728e00b62d9b3fe5727d4d5a78fbc"
server_key_version: "encryptionkey@0"
I20260807 19:46:46.673450 18738 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 19:46:46.676818 18752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:46.678102 18738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:46:46.678247 18738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "f8d464853a31407782fca51ad788727c"
format_stamp: "Formatted at 2026-08-07 19:46:46 on dist-test-slave-vq2q"
server_key: "4e69efb6a587948ed203bae606cbd1d5"
server_key_iv: "17b728e00b62d9b3fe5727d4d5a78fbc"
server_key_version: "encryptionkey@0"
I20260807 19:46:46.678457 18738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:46.699700 18738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:46.700549 18738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:46.700856 18738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:46.701886 18738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:46.703442 18738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:46.703543 18738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:46.703651 18738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:46.703727 18738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:46.734860 18738 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:36249
I20260807 19:46:46.734966 18876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:36249 every 8 connection(s)
I20260807 19:46:46.736092 18738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:46:46.739476 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 18738
I20260807 19:46:46.739652 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:46:46.740084 31552 external_mini_cluster.cc:1428] Setting key 6443c59c8fadbea4f82990cc2ce1fbff
I20260807 19:46:46.742882 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41071
--builtin_ntp_servers=127.30.208.20:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:46:46.748854 18879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:46.738617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:45565 (local address 127.30.208.62:41071)
0807 19:46:46.739121 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:46.739128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:46.740423 (+ 1295us) server_negotiation.cc:407] Connection header received
0807 19:46:46.742356 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:46.742362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:46.742442 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:46.742560 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:46.743654 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:46.744634 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:46.746205 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:46.746732 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:46.746851 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:46.748103 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:46.748109 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:46.748113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:46.748133 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:46.748180 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:46.748194 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:46.748285 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:46.748444 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 19:46:46.748541 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":231,"threads_started":1}
I20260807 19:46:46.749760 18880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:46.739241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:41071 (local address 127.30.208.1:45565)
0807 19:46:46.740259 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:46.740302 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:46.742112 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:46.742781 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:46.742792 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:46.742911 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:46.743488 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:46.743504 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:46.744796 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:46.744800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:46.746069 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:46.746082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:46.746950 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:46.746955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:46.747158 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:46.747777 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:46:46.747779 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:46.747815 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:46:46.747917 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:46.748277 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:46.748283 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:46.748288 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 19:46:46.748417 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 19:46:46.748696 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":317,"threads_started":1}
I20260807 19:46:46.751206 18877 heartbeater.cc:347] Connected to a master server at 127.30.208.62:41071
I20260807 19:46:46.751529 18877 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:46.752404 18877 heartbeater.cc:511] Master 127.30.208.62:41071 requested a full tablet report, sending...
I20260807 19:46:46.754519 18674 ts_manager.cc:194] Registered new tserver with Master: f8d464853a31407782fca51ad788727c (127.30.208.1:36249)
I20260807 19:46:46.756516 18674 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.208.1:45565
W20260807 19:46:47.081199 18881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:47.081528 18881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:47.081638 18881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:47.088994 18881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:47.089108 18881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:47.089295 18881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:46:47.098529 18881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:47.099855 18881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:47.101712 18881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:47.114692 18887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:47.114838 18891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:47.114900 18888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:47.115931 18881 server_base.cc:1034] running on GCE node
I20260807 19:46:47.116549 18881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:47.117584 18881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:47.118808 18881 hybrid_clock.cc:648] HybridClock initialized: now 1786132007118746 us; error 56 us; skew 500 ppm
I20260807 19:46:47.119130 18881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:46:47.121490 18881 webserver.cc:460] Webserver started at http://127.30.208.2:45159/ using document root <none> and password file <none>
I20260807 19:46:47.122174 18881 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:47.122282 18881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:47.122584 18881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:47.125516 18881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "d3a2a153cbeb4407be0d625c7f45e21a"
format_stamp: "Formatted at 2026-08-07 19:46:47 on dist-test-slave-vq2q"
server_key: "37c2e535922fd7bef9ae10185c213734"
server_key_iv: "c5fd04623c8b770e4a875f3f51993dcd"
server_key_version: "encryptionkey@0"
I20260807 19:46:47.126329 18881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "d3a2a153cbeb4407be0d625c7f45e21a"
format_stamp: "Formatted at 2026-08-07 19:46:47 on dist-test-slave-vq2q"
server_key: "37c2e535922fd7bef9ae10185c213734"
server_key_iv: "c5fd04623c8b770e4a875f3f51993dcd"
server_key_version: "encryptionkey@0"
I20260807 19:46:47.131553 18881 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:46:47.135319 18898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:47.136664 18881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:46:47.136849 18881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "d3a2a153cbeb4407be0d625c7f45e21a"
format_stamp: "Formatted at 2026-08-07 19:46:47 on dist-test-slave-vq2q"
server_key: "37c2e535922fd7bef9ae10185c213734"
server_key_iv: "c5fd04623c8b770e4a875f3f51993dcd"
server_key_version: "encryptionkey@0"
I20260807 19:46:47.137068 18881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:47.152711 18881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:47.153541 18881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:47.153829 18881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:47.154748 18881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:47.156253 18881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:47.156348 18881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:47.156461 18881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:47.156528 18881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:47.188477 18881 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:42479
I20260807 19:46:47.188548 19020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:42479 every 8 connection(s)
I20260807 19:46:47.189898 18881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:46:47.194386 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 18881
I20260807 19:46:47.194566 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:46:47.195007 31552 external_mini_cluster.cc:1428] Setting key 1de8cf1fb805fd94d3843a32760b1d1e
I20260807 19:46:47.197715 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41071
--builtin_ntp_servers=127.30.208.20:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:46:47.204926 18879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.192584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:56965 (local address 127.30.208.62:41071)
0807 19:46:47.192793 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:47.192799 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:47.194392 (+ 1593us) server_negotiation.cc:407] Connection header received
0807 19:46:47.196909 (+ 2517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:47.196914 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:47.196997 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:47.197106 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:47.198441 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.199511 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.201871 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.202390 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.202507 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.204117 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:47.204123 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:47.204127 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:47.204148 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:47.204189 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:47.204203 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:47.204289 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:47.204522 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 19:46:47.204619 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:46:47.206336 19023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.193080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:41071 (local address 127.30.208.2:56965)
0807 19:46:47.194201 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:47.194254 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:47.196541 (+ 2287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:47.197410 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:47.197424 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:47.197544 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:47.198253 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.198274 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.199694 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.199699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.201694 (+ 1995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.201710 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.202631 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.202638 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.202924 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.203727 (+ 803us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:46:47.203731 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:47.203786 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:46:47.203886 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:47.204303 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:47.204313 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:47.204321 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 19:46:47.204484 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 19:46:47.204898 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":310,"threads_started":1}
I20260807 19:46:47.207831 19021 heartbeater.cc:347] Connected to a master server at 127.30.208.62:41071
I20260807 19:46:47.208148 19021 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:47.209026 19021 heartbeater.cc:511] Master 127.30.208.62:41071 requested a full tablet report, sending...
I20260807 19:46:47.210522 18674 ts_manager.cc:194] Registered new tserver with Master: d3a2a153cbeb4407be0d625c7f45e21a (127.30.208.2:42479)
I20260807 19:46:47.211313 18674 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.208.2:56965
W20260807 19:46:47.533816 19024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:47.534101 19024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:47.534226 19024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:47.541664 19024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:47.541776 19024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:47.541924 19024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:46:47.551178 19024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:41071
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:47.552498 19024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:47.554272 19024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:47.566907 19032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:47.566947 19030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:47.566974 19029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:47.567803 19024 server_base.cc:1034] running on GCE node
I20260807 19:46:47.568361 19024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:47.569265 19024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:47.570466 19024 hybrid_clock.cc:648] HybridClock initialized: now 1786132007570427 us; error 32 us; skew 500 ppm
I20260807 19:46:47.570763 19024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:46:47.572978 19024 webserver.cc:460] Webserver started at http://127.30.208.3:41663/ using document root <none> and password file <none>
I20260807 19:46:47.573685 19024 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:47.573781 19024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:47.574075 19024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:47.576877 19024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "fb81975c62cf422e8be897c3edde2263"
format_stamp: "Formatted at 2026-08-07 19:46:47 on dist-test-slave-vq2q"
server_key: "a200848d2c55cc5f61218f304873a8d7"
server_key_iv: "6c9cc18ffbcd03ed9e8a28ba9ccc6ab0"
server_key_version: "encryptionkey@0"
I20260807 19:46:47.577777 19024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "fb81975c62cf422e8be897c3edde2263"
format_stamp: "Formatted at 2026-08-07 19:46:47 on dist-test-slave-vq2q"
server_key: "a200848d2c55cc5f61218f304873a8d7"
server_key_iv: "6c9cc18ffbcd03ed9e8a28ba9ccc6ab0"
server_key_version: "encryptionkey@0"
I20260807 19:46:47.582711 19024 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:46:47.586090 19038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:47.587425 19024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:46:47.587577 19024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "fb81975c62cf422e8be897c3edde2263"
format_stamp: "Formatted at 2026-08-07 19:46:47 on dist-test-slave-vq2q"
server_key: "a200848d2c55cc5f61218f304873a8d7"
server_key_iv: "6c9cc18ffbcd03ed9e8a28ba9ccc6ab0"
server_key_version: "encryptionkey@0"
I20260807 19:46:47.587772 19024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:47.606779 19024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:47.607640 19024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:47.607947 19024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:47.609019 19024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:47.610605 19024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:47.610699 19024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:47.610805 19024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:47.610890 19024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:47.642277 19024 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:39963
I20260807 19:46:47.642359 19165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:39963 every 8 connection(s)
I20260807 19:46:47.643577 19024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:46:47.649029 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 19024
I20260807 19:46:47.649255 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:46:47.649822 31552 external_mini_cluster.cc:1428] Setting key 882aaea7067fe6754b0ba51a625982fd
I20260807 19:46:47.656327 18879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.646253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:50913 (local address 127.30.208.62:41071)
0807 19:46:47.646436 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:47.646440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:47.647788 (+ 1348us) server_negotiation.cc:407] Connection header received
0807 19:46:47.650062 (+ 2274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:47.650067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:47.650143 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:47.650230 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:47.651455 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.652239 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.653913 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.654337 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.654432 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.655712 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:47.655717 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:47.655720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:47.655738 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:47.655769 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:47.655781 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:47.655856 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:47.656010 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 19:46:47.656088 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:46:47.657493 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.646707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:41071 (local address 127.30.208.3:50913)
0807 19:46:47.647626 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:47.647665 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:47.649746 (+ 2081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:47.650459 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:47.650468 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:47.650557 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:47.651276 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.651305 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.652401 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.652405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.653771 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.653785 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.654553 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.654558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.654776 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.655401 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:46:47.655404 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:47.655444 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:46:47.655530 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:47.655853 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:47.655858 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:47.655864 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:46:47.655999 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 19:46:47.656317 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":272,"threads_started":1}
I20260807 19:46:47.658849 19166 heartbeater.cc:347] Connected to a master server at 127.30.208.62:41071
I20260807 19:46:47.659137 19166 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:47.659878 19166 heartbeater.cc:511] Master 127.30.208.62:41071 requested a full tablet report, sending...
I20260807 19:46:47.661294 18674 ts_manager.cc:194] Registered new tserver with Master: fb81975c62cf422e8be897c3edde2263 (127.30.208.3:39963)
I20260807 19:46:47.662106 18674 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.208.3:50913
I20260807 19:46:47.664885 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:46:47.675011 18879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.668928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.30.208.62:41071)
0807 19:46:47.669122 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:47.669129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:47.669550 (+ 421us) server_negotiation.cc:407] Connection header received
0807 19:46:47.669672 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:47.669675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:47.669746 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:47.669824 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:47.670273 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.670958 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.672455 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.672914 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.673094 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.674261 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:47.674266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:47.674269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:47.674289 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:47.674323 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:47.674336 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:47.674407 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:47.674556 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 19:46:47.674644 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 19:46:47.679153 18674 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:46:47.686209 18674 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:46:47.708786 19182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.698512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:42479 (local address 127.0.0.1:43108)
0807 19:46:47.699826 (+ 1314us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:47.699844 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:47.699978 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:47.701269 (+ 1291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:47.701277 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:47.701308 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:47.701758 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.701772 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.703061 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.703066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.705609 (+ 2543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.705621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.707833 (+ 2212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.707839 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.708050 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.708102 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 19:46:47.708234 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 19:46:47.708434 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1166,"thread_start_us":1348,"threads_started":1}
I20260807 19:46:47.708850 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.699174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43108 (local address 127.30.208.2:42479)
0807 19:46:47.700626 (+ 1452us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:47.700632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:47.700661 (+ 29us) server_negotiation.cc:407] Connection header received
0807 19:46:47.700761 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:47.700767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:47.701004 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:47.701223 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:47.701939 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.702914 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.705814 (+ 2900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.707003 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.707478 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.707715 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:47.708316 (+ 601us) server_negotiation.cc:299] Negotiation successful
0807 19:46:47.708494 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1305,"thread_start_us":1316,"threads_started":1}
I20260807 19:46:47.709334 19179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.697978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:36249 (local address 127.0.0.1:35818)
0807 19:46:47.698701 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:47.698743 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:47.698923 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:47.701645 (+ 2722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:47.701650 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:47.701684 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:47.703422 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.703439 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.704815 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.704819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.706973 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.706990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.708550 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.708557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.708741 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.708778 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 19:46:47.708849 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 19:46:47.708979 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":21,"thread_start_us":337,"threads_started":1}
I20260807 19:46:47.710363 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.699608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56934 (local address 127.30.208.3:39963)
0807 19:46:47.700620 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:47.700629 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:47.700664 (+ 35us) server_negotiation.cc:407] Connection header received
0807 19:46:47.700759 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:47.700767 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:47.701007 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:47.701231 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:47.703645 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.705251 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.707681 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.709026 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.709149 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.709443 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:47.709892 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 19:46:47.710051 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":844,"thread_start_us":690,"threads_started":1}
I20260807 19:46:47.710379 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.698227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35818 (local address 127.30.208.1:36249)
0807 19:46:47.701054 (+ 2827us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:47.701060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:47.701083 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:46:47.701164 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:47.701170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:47.701425 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:47.702019 (+ 594us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:47.703641 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.704674 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.707211 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.708339 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.709558 (+ 1219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.709787 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:47.709859 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 19:46:47.710028 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2683,"thread_start_us":2581,"threads_started":1}
I20260807 19:46:47.710636 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.699104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:39963 (local address 127.0.0.1:56934)
0807 19:46:47.700101 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:47.700115 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:47.700204 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:47.701516 (+ 1312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:47.701520 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:47.701556 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:47.703382 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.703400 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.705381 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.705390 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.707490 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.707503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.709378 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.709384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.709688 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.709733 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 19:46:47.710176 (+ 443us) client_negotiation.cc:240] Negotiation successful
0807 19:46:47.710316 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"mutex_wait_us":4,"thread_start_us":415,"threads_started":1}
I20260807 19:46:47.713446 18806 tablet_service.cc:1467] Processing CreateTablet for tablet 271427817d2d475cb88889158a83c58d (DEFAULT_TABLE table=test-table [id=158da4cab0da4ac884d5e1005c15c01e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:47.713435 18950 tablet_service.cc:1467] Processing CreateTablet for tablet 271427817d2d475cb88889158a83c58d (DEFAULT_TABLE table=test-table [id=158da4cab0da4ac884d5e1005c15c01e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:47.713979 19090 tablet_service.cc:1467] Processing CreateTablet for tablet 271427817d2d475cb88889158a83c58d (DEFAULT_TABLE table=test-table [id=158da4cab0da4ac884d5e1005c15c01e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:47.714609 18806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271427817d2d475cb88889158a83c58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:47.714735 18950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271427817d2d475cb88889158a83c58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:47.715214 19090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 271427817d2d475cb88889158a83c58d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:47.725939 19186 tablet_bootstrap.cc:492] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c: Bootstrap starting.
I20260807 19:46:47.729097 19187 tablet_bootstrap.cc:492] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a: Bootstrap starting.
I20260807 19:46:47.730937 19186 tablet_bootstrap.cc:654] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:47.732260 19186 log.cc:824] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:47.732864 19187 tablet_bootstrap.cc:654] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:47.734162 19187 log.cc:824] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:47.735873 19186 tablet_bootstrap.cc:492] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c: No bootstrap required, opened a new log
I20260807 19:46:47.736179 19186 ts_tablet_manager.cc:1397] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 19:46:47.737789 19188 tablet_bootstrap.cc:492] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263: Bootstrap starting.
I20260807 19:46:47.740099 19187 tablet_bootstrap.cc:492] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a: No bootstrap required, opened a new log
I20260807 19:46:47.740371 19187 ts_tablet_manager.cc:1397] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260807 19:46:47.741442 19188 tablet_bootstrap.cc:654] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:47.742516 19188 log.cc:824] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:47.748840 19186 raft_consensus.cc:348] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d464853a31407782fca51ad788727c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36249 } } peers { permanent_uuid: "fb81975c62cf422e8be897c3edde2263" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39963 } } peers { permanent_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 42479 } }
I20260807 19:46:47.749508 19186 raft_consensus.cc:374] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:47.749670 19186 raft_consensus.cc:729] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8d464853a31407782fca51ad788727c, State: Initialized, Role: FOLLOWER
I20260807 19:46:47.750519 19186 consensus_queue.cc:260] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d464853a31407782fca51ad788727c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36249 } } peers { permanent_uuid: "fb81975c62cf422e8be897c3edde2263" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39963 } } peers { permanent_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 42479 } }
I20260807 19:46:47.750458 19187 raft_consensus.cc:348] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d464853a31407782fca51ad788727c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36249 } } peers { permanent_uuid: "fb81975c62cf422e8be897c3edde2263" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39963 } } peers { permanent_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 42479 } }
I20260807 19:46:47.750898 19187 raft_consensus.cc:374] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:47.751034 19187 raft_consensus.cc:729] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3a2a153cbeb4407be0d625c7f45e21a, State: Initialized, Role: FOLLOWER
I20260807 19:46:47.751757 19187 consensus_queue.cc:260] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d464853a31407782fca51ad788727c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36249 } } peers { permanent_uuid: "fb81975c62cf422e8be897c3edde2263" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39963 } } peers { permanent_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 42479 } }
I20260807 19:46:47.752388 19188 tablet_bootstrap.cc:492] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263: No bootstrap required, opened a new log
I20260807 19:46:47.752774 19188 ts_tablet_manager.cc:1397] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.009s
I20260807 19:46:47.752936 19021 heartbeater.cc:503] Master 127.30.208.62:41071 was elected leader, sending a full tablet report...
I20260807 19:46:47.754853 19187 ts_tablet_manager.cc:1428] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a: Time spent starting tablet: real 0.014s user 0.009s sys 0.006s
I20260807 19:46:47.755781 18877 heartbeater.cc:503] Master 127.30.208.62:41071 was elected leader, sending a full tablet report...
I20260807 19:46:47.756382 19186 ts_tablet_manager.cc:1428] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c: Time spent starting tablet: real 0.020s user 0.020s sys 0.000s
I20260807 19:46:47.762780 19188 raft_consensus.cc:348] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d464853a31407782fca51ad788727c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36249 } } peers { permanent_uuid: "fb81975c62cf422e8be897c3edde2263" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39963 } } peers { permanent_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 42479 } }
I20260807 19:46:47.763199 19188 raft_consensus.cc:374] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:47.763315 19188 raft_consensus.cc:729] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb81975c62cf422e8be897c3edde2263, State: Initialized, Role: FOLLOWER
I20260807 19:46:47.763810 19188 consensus_queue.cc:260] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d464853a31407782fca51ad788727c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36249 } } peers { permanent_uuid: "fb81975c62cf422e8be897c3edde2263" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39963 } } peers { permanent_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 42479 } }
I20260807 19:46:47.764743 19166 heartbeater.cc:503] Master 127.30.208.62:41071 was elected leader, sending a full tablet report...
I20260807 19:46:47.765376 19188 ts_tablet_manager.cc:1428] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260807 19:46:47.860411 19194 raft_consensus.cc:482] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:47.860702 19194 raft_consensus.cc:504] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d464853a31407782fca51ad788727c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36249 } } peers { permanent_uuid: "fb81975c62cf422e8be897c3edde2263" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39963 } } peers { permanent_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 42479 } }
I20260807 19:46:47.862351 19194 leader_election.cc:288] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8d464853a31407782fca51ad788727c (127.30.208.1:36249), d3a2a153cbeb4407be0d625c7f45e21a (127.30.208.2:42479)
I20260807 19:46:47.868388 19168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.862928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:36249 (local address 127.30.208.3:38801)
0807 19:46:47.863140 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:47.863158 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:47.863288 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:47.863701 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:47.863706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:47.863738 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:47.863950 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.863961 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.865262 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.865268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.866674 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.866690 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.867868 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.867872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.867988 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.868012 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 19:46:47.868082 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 19:46:47.868171 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 19:46:47.868497 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.862777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:38801 (local address 127.30.208.1:36249)
0807 19:46:47.862996 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:47.863002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:47.863249 (+ 247us) server_negotiation.cc:407] Connection header received
0807 19:46:47.863421 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:47.863428 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:47.863525 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:47.863639 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:47.864097 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.865130 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.866820 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.867739 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.867894 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.867960 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:47.868120 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 19:46:47.868241 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 19:46:47.869388 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.862921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:43517 (local address 127.30.208.2:42479)
0807 19:46:47.863103 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:47.863108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:47.863837 (+ 729us) server_negotiation.cc:407] Connection header received
0807 19:46:47.864036 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:47.864042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:47.864130 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:47.864224 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:47.864891 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.865677 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.867429 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.868563 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.868723 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.868787 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:47.869002 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 19:46:47.869123 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:46:47.869426 19195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.862944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:42479 (local address 127.30.208.3:43517)
0807 19:46:47.863739 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:47.863754 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:47.863898 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:47.864271 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:47.864277 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:47.864333 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:47.864696 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.864708 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.865872 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.865875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.867256 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.867269 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.868705 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.868720 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.868845 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.868868 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 19:46:47.868957 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 19:46:47.869071 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":84,"thread_start_us":422,"threads_started":1}
I20260807 19:46:47.870095 18828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271427817d2d475cb88889158a83c58d" candidate_uuid: "fb81975c62cf422e8be897c3edde2263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8d464853a31407782fca51ad788727c" is_pre_election: true
I20260807 19:46:47.870499 19193 raft_consensus.cc:482] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:47.870656 18828 raft_consensus.cc:2454] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb81975c62cf422e8be897c3edde2263 in term 0.
I20260807 19:46:47.870715 18971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271427817d2d475cb88889158a83c58d" candidate_uuid: "fb81975c62cf422e8be897c3edde2263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" is_pre_election: true
I20260807 19:46:47.870803 19193 raft_consensus.cc:504] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d464853a31407782fca51ad788727c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36249 } } peers { permanent_uuid: "fb81975c62cf422e8be897c3edde2263" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39963 } } peers { permanent_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 42479 } }
I20260807 19:46:47.871517 19041 leader_election.cc:302] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f8d464853a31407782fca51ad788727c, fb81975c62cf422e8be897c3edde2263; no voters:
I20260807 19:46:47.871560 18971 raft_consensus.cc:2454] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb81975c62cf422e8be897c3edde2263 in term 0.
I20260807 19:46:47.871835 19194 raft_consensus.cc:2781] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:46:47.871989 19194 raft_consensus.cc:482] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:46:47.872103 19194 raft_consensus.cc:3037] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:47.872712 19193 leader_election.cc:288] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8d464853a31407782fca51ad788727c (127.30.208.1:36249), fb81975c62cf422e8be897c3edde2263 (127.30.208.3:39963)
I20260807 19:46:47.875393 19194 raft_consensus.cc:504] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d464853a31407782fca51ad788727c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36249 } } peers { permanent_uuid: "fb81975c62cf422e8be897c3edde2263" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39963 } } peers { permanent_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 42479 } }
I20260807 19:46:47.876118 19194 leader_election.cc:288] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [CANDIDATE]: Term 1 election: Requested vote from peers f8d464853a31407782fca51ad788727c (127.30.208.1:36249), d3a2a153cbeb4407be0d625c7f45e21a (127.30.208.2:42479)
I20260807 19:46:47.876554 18828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271427817d2d475cb88889158a83c58d" candidate_uuid: "fb81975c62cf422e8be897c3edde2263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8d464853a31407782fca51ad788727c"
I20260807 19:46:47.876768 18828 raft_consensus.cc:3037] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:47.877792 18971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271427817d2d475cb88889158a83c58d" candidate_uuid: "fb81975c62cf422e8be897c3edde2263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3a2a153cbeb4407be0d625c7f45e21a"
I20260807 19:46:47.878005 18971 raft_consensus.cc:3037] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:47.880100 18828 raft_consensus.cc:2454] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb81975c62cf422e8be897c3edde2263 in term 1.
I20260807 19:46:47.880564 19041 leader_election.cc:302] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f8d464853a31407782fca51ad788727c, fb81975c62cf422e8be897c3edde2263; no voters:
I20260807 19:46:47.880923 19194 raft_consensus.cc:2781] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:46:47.881641 18971 raft_consensus.cc:2454] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb81975c62cf422e8be897c3edde2263 in term 1.
I20260807 19:46:47.882495 19194 raft_consensus.cc:686] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [term 1 LEADER]: Becoming Leader. State: Replica: fb81975c62cf422e8be897c3edde2263, State: Running, Role: LEADER
I20260807 19:46:47.883162 19194 consensus_queue.cc:237] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d464853a31407782fca51ad788727c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36249 } } peers { permanent_uuid: "fb81975c62cf422e8be897c3edde2263" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39963 } } peers { permanent_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 42479 } }
I20260807 19:46:47.883767 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.873009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:35215 (local address 127.30.208.1:36249)
0807 19:46:47.873166 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:47.873170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:47.876204 (+ 3034us) server_negotiation.cc:407] Connection header received
0807 19:46:47.876489 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:47.876495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:47.876578 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:47.876624 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:47.877364 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.878526 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.881393 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.882619 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.882737 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.882806 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:47.883379 (+ 573us) server_negotiation.cc:299] Negotiation successful
0807 19:46:47.883514 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 19:46:47.883793 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.872959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:36249 (local address 127.30.208.2:35215)
0807 19:46:47.876119 (+ 3160us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:47.876137 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:47.876335 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:47.876784 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:47.876789 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:47.876814 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:47.877159 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.877168 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.878709 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.878715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.881137 (+ 2422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.881153 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.882996 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.883003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.883174 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.883221 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 19:46:47.883305 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 19:46:47.883435 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3033,"mutex_wait_us":101,"thread_start_us":2872,"threads_started":1}
I20260807 19:46:47.884415 18828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271427817d2d475cb88889158a83c58d" candidate_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8d464853a31407782fca51ad788727c" is_pre_election: true
I20260807 19:46:47.884737 18828 raft_consensus.cc:2379] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d3a2a153cbeb4407be0d625c7f45e21a in current term 1: Already voted for candidate fb81975c62cf422e8be897c3edde2263 in this term.
I20260807 19:46:47.884850 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.876423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:39959 (local address 127.30.208.3:39963)
0807 19:46:47.876590 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:47.876595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:47.877735 (+ 1140us) server_negotiation.cc:407] Connection header received
0807 19:46:47.877986 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:47.877992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:47.878058 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:47.878113 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:47.878999 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.880045 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.882732 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.883979 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.884122 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.884192 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:47.884499 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 19:46:47.884622 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:46:47.884914 19198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.876724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:39963 (local address 127.30.208.2:39959)
0807 19:46:47.877636 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:47.877654 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:47.877845 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:47.878281 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:47.878286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:47.878312 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:47.878788 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.878799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.880224 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.880229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.882508 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:47.882524 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.884202 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.884221 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:47.884354 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.884376 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 19:46:47.884457 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 19:46:47.884577 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":1,"thread_start_us":608,"threads_started":1}
I20260807 19:46:47.885785 19111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "271427817d2d475cb88889158a83c58d" candidate_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb81975c62cf422e8be897c3edde2263" is_pre_election: true
I20260807 19:46:47.886847 18901 leader_election.cc:302] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d3a2a153cbeb4407be0d625c7f45e21a; no voters: f8d464853a31407782fca51ad788727c, fb81975c62cf422e8be897c3edde2263
I20260807 19:46:47.887298 19193 raft_consensus.cc:2726] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 19:46:47.889317 18674 catalog_manager.cc:5495] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb81975c62cf422e8be897c3edde2263 (127.30.208.3). New cstate: current_term: 1 leader_uuid: "fb81975c62cf422e8be897c3edde2263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d464853a31407782fca51ad788727c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb81975c62cf422e8be897c3edde2263" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 39963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 42479 } health_report { overall_health: UNKNOWN } } }
W20260807 19:46:47.895140 19167 tablet.cc:2369] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 19:46:47.942081 19022 tablet.cc:2369] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:46:47.948359 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.943138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56950 (local address 127.30.208.3:39963)
0807 19:46:47.943347 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:47.943352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:47.943397 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:46:47.943586 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:47.943590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:47.943712 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:47.943856 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:46:47.944199 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.944954 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.946398 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:47.946813 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:47.946914 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:47.947319 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:46:47.947803 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:46:47.947929 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:47.948012 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 19:46:47.948073 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:46:47.956382 18828 raft_consensus.cc:1261] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c [term 1 FOLLOWER]: Refusing update from remote peer fb81975c62cf422e8be897c3edde2263: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:46:47.956398 18971 raft_consensus.cc:1261] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [term 1 FOLLOWER]: Refusing update from remote peer fb81975c62cf422e8be897c3edde2263: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:46:47.957369 19199 consensus_queue.cc:1035] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8d464853a31407782fca51ad788727c" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:46:47.957795 19194 consensus_queue.cc:1035] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3a2a153cbeb4407be0d625c7f45e21a" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 42479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:46:47.983310 18674 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:46:47.983594 18674 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:46:47.987361 18674 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d71e9e35baef4de3b57dbc68d3c1f28b: Sending DeleteTablet for 3 replicas of tablet 271427817d2d475cb88889158a83c58d
I20260807 19:46:47.988720 18806 tablet_service.cc:1514] Processing DeleteTablet for tablet 271427817d2d475cb88889158a83c58d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:47 UTC) from {username='slave'} at 127.0.0.1:35818
I20260807 19:46:47.989326 19090 tablet_service.cc:1514] Processing DeleteTablet for tablet 271427817d2d475cb88889158a83c58d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:47 UTC) from {username='slave'} at 127.0.0.1:56934
I20260807 19:46:47.989681 18950 tablet_service.cc:1514] Processing DeleteTablet for tablet 271427817d2d475cb88889158a83c58d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:47 UTC) from {username='slave'} at 127.0.0.1:43108
I20260807 19:46:47.990336 19216 tablet_replica.cc:330] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263: stopping tablet replica
I20260807 19:46:47.990525 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 18738
I20260807 19:46:47.990845 19216 raft_consensus.cc:2229] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:46:47.991619 19216 raft_consensus.cc:2258] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:47.994396 19216 ts_tablet_manager.cc:1905] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:48.001545 19217 tablet_replica.cc:330] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a: stopping tablet replica
I20260807 19:46:48.001605 19214 tablet_replica.cc:330] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c: stopping tablet replica
I20260807 19:46:48.002074 19217 raft_consensus.cc:2229] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:46:48.002074 19214 raft_consensus.cc:2229] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:46:48.002614 19217 raft_consensus.cc:2258] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:48.002624 19214 raft_consensus.cc:2258] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:48.004787 19217 ts_tablet_manager.cc:1905] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:48.004794 19214 ts_tablet_manager.cc:1905] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:48.011755 19214 ts_tablet_manager.cc:1918] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:48.011973 19217 ts_tablet_manager.cc:1918] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:48.012034 19214 log.cc:1196] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/271427817d2d475cb88889158a83c58d
I20260807 19:46:48.012163 19217 log.cc:1196] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/271427817d2d475cb88889158a83c58d
I20260807 19:46:48.012641 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:47.992237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35824 (local address 127.30.208.1:36249)
0807 19:46:47.992806 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:47.992813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:48.002593 (+ 9780us) server_negotiation.cc:407] Connection header received
0807 19:46:48.002997 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:48.003003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:48.003128 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:48.003211 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:48.003965 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:48.005068 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:48.007278 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:48.007928 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:48.008031 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:48.010694 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:48.010707 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:48.010726 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:48.011608 (+ 882us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:48.011689 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:48.011717 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:48.011813 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:48.012110 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 19:46:48.012254 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418}
I20260807 19:46:48.012754 19214 ts_tablet_manager.cc:1939] T 271427817d2d475cb88889158a83c58d P f8d464853a31407782fca51ad788727c: Deleting consensus metadata
I20260807 19:46:48.012981 19217 ts_tablet_manager.cc:1939] T 271427817d2d475cb88889158a83c58d P d3a2a153cbeb4407be0d625c7f45e21a: Deleting consensus metadata
I20260807 19:46:48.013919 18872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:48.014547 19216 ts_tablet_manager.cc:1918] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:48.014786 19216 log.cc:1196] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/271427817d2d475cb88889158a83c58d
I20260807 19:46:48.014775 18661 catalog_manager.cc:4841] TS d3a2a153cbeb4407be0d625c7f45e21a (127.30.208.2:42479): tablet 271427817d2d475cb88889158a83c58d (table test-table [id=158da4cab0da4ac884d5e1005c15c01e]) successfully deleted
I20260807 19:46:48.015380 19216 ts_tablet_manager.cc:1939] T 271427817d2d475cb88889158a83c58d P fb81975c62cf422e8be897c3edde2263: Deleting consensus metadata
I20260807 19:46:48.017331 18660 catalog_manager.cc:4841] TS fb81975c62cf422e8be897c3edde2263 (127.30.208.3:39963): tablet 271427817d2d475cb88889158a83c58d (table test-table [id=158da4cab0da4ac884d5e1005c15c01e]) successfully deleted
I20260807 19:46:48.105969 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 18738
I20260807 19:46:48.123467 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 18881
W20260807 19:46:48.123718 18660 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.208.1:36249 (error 108)
I20260807 19:46:48.126078 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:48.125548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:36249 (local address 127.0.0.1:35830)
0807 19:46:48.125759 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:48.125900 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:36249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 19:46:48.126439 18660 catalog_manager.cc:4584] TS f8d464853a31407782fca51ad788727c (127.30.208.1:36249): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 271427817d2d475cb88889158a83c58d: Network error: Client connection negotiation failed: client connection to 127.30.208.1:36249: connect: Connection refused (error 111)
I20260807 19:46:48.130630 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:48.124325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43118 (local address 127.30.208.2:42479)
0807 19:46:48.124510 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:48.124516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:48.124539 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:46:48.124605 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:48.124610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:48.124756 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:48.124873 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:48.125350 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:48.126318 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:48.127651 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:48.128079 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:48.128193 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:48.129288 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:48.129299 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:48.129316 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:48.129905 (+ 589us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:48.129953 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:48.129973 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:48.130080 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:48.130208 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 19:46:48.130320 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:46:48.131372 19015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:48.180279 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:48.179714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:36249 (local address 127.0.0.1:35842)
0807 19:46:48.179955 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:48.180075 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:36249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 19:46:48.181529 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:48.181042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:36249 (local address 127.0.0.1:35856)
0807 19:46:48.181265 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:48.181359 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:36249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 19:46:48.223013 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 18881
I20260807 19:46:48.240969 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 19024
I20260807 19:46:48.242166 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:48.241610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:36249 (local address 127.0.0.1:35870)
0807 19:46:48.241906 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:48.241996 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:36249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
I20260807 19:46:48.243361 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:48.242936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:36249 (local address 127.0.0.1:35886)
0807 19:46:48.243163 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:48.243237 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:36249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260807 19:46:48.249389 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:48.241747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56962 (local address 127.30.208.3:39963)
0807 19:46:48.241908 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:48.241913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:48.242079 (+ 166us) server_negotiation.cc:407] Connection header received
0807 19:46:48.242277 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:48.242283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:48.242423 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:48.242536 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:48.243134 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:48.244324 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:48.245729 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:48.246407 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:48.246584 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:48.247613 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:48.247624 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:48.247643 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:48.248525 (+ 882us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:48.248592 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:48.248619 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:48.248737 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:48.248882 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 19:46:48.248994 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:46:48.250253 19161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:48.324079 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:48.323517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:36249 (local address 127.0.0.1:35894)
0807 19:46:48.323762 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:48.323875 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:36249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 19:46:48.325294 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:48.324776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:36249 (local address 127.0.0.1:35898)
0807 19:46:48.325018 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:48.325105 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:36249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260807 19:46:48.370947 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 19024
I20260807 19:46:48.390610 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 18642
I20260807 19:46:48.391829 18706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:48.501279 19181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:48.500715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:36249 (local address 127.0.0.1:35910)
0807 19:46:48.500947 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:48.501061 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:36249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 19:46:48.501279 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 18642
2026-08-07T19:46:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2830 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:46:48 dist-test-slave-vq2q krb5kdc[19228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:46:48 dist-test-slave-vq2q krb5kdc[19228](info): set up 2 sockets
Aug 07 19:46:48 dist-test-slave-vq2q krb5kdc[19228](info): commencing operation
krb5kdc: starting...
W20260807 19:46:49.588126 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.005s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:46:49 dist-test-slave-vq2q krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:46:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:46:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:46:49.888347 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:46491
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:40521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:46491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:50.228638 19242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:50.228956 19242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:50.229091 19242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:50.236470 19242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:46:50.236581 19242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:50.236653 19242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:50.236734 19242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:46:50.236794 19242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:46:50.247076 19242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:40521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:46491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:46491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:50.248427 19242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:50.249871 19242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:50.261482 19249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:50.261425 19248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:50.261518 19251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:50.262331 19242 server_base.cc:1034] running on GCE node
I20260807 19:46:50.263147 19242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:50.264667 19242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:50.265897 19242 hybrid_clock.cc:648] HybridClock initialized: now 1786132010265815 us; error 44 us; skew 500 ppm
I20260807 19:46:50.266255 19242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:50 dist-test-slave-vq2q krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:50.269512 19242 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:46:50.271147 19242 webserver.cc:460] Webserver started at http://127.30.208.62:44037/ using document root <none> and password file <none>
I20260807 19:46:50.271848 19242 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:50.271977 19242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:50.272274 19242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:50.275449 19242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "c76b22f6b9fb4feda93a165a24d6f60f"
format_stamp: "Formatted at 2026-08-07 19:46:50 on dist-test-slave-vq2q"
server_key: "414cc9a9a6a684affd639366d10711c0"
server_key_iv: "d49ebf528439ec422864732e77b33f6f"
server_key_version: "encryptionkey@0"
I20260807 19:46:50.276189 19242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "c76b22f6b9fb4feda93a165a24d6f60f"
format_stamp: "Formatted at 2026-08-07 19:46:50 on dist-test-slave-vq2q"
server_key: "414cc9a9a6a684affd639366d10711c0"
server_key_iv: "d49ebf528439ec422864732e77b33f6f"
server_key_version: "encryptionkey@0"
I20260807 19:46:50.281379 19242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 19:46:50.284940 19258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:50.286362 19242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:46:50.286504 19242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "c76b22f6b9fb4feda93a165a24d6f60f"
format_stamp: "Formatted at 2026-08-07 19:46:50 on dist-test-slave-vq2q"
server_key: "414cc9a9a6a684affd639366d10711c0"
server_key_iv: "d49ebf528439ec422864732e77b33f6f"
server_key_version: "encryptionkey@0"
I20260807 19:46:50.286689 19242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:50.308091 19242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:50.312387 19242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:50.312633 19242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:50.329384 19242 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:46491
I20260807 19:46:50.329396 19314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:46491 every 8 connection(s)
I20260807 19:46:50.330790 19242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:46:50.334517 19316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:50.336524 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 19242
I20260807 19:46:50.336740 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:46:50.337236 31552 external_mini_cluster.cc:1428] Setting key 6b66e3838c8cae85d749b94cfb2d3bea
I20260807 19:46:50.346463 19316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f: Bootstrap starting.
I20260807 19:46:50.350334 19316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:50.351666 19316 log.cc:824] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:46:50 dist-test-slave-vq2q krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:50.354710 19316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f: No bootstrap required, opened a new log
I20260807 19:46:50.360575 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:50.339024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57250 (local address 127.30.208.62:46491)
0807 19:46:50.339771 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:50.339783 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:50.339829 (+ 46us) server_negotiation.cc:407] Connection header received
0807 19:46:50.341532 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:50.341555 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:50.341930 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:50.342224 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:50.344071 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:50.345470 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:50.347297 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:50.347988 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:50.348134 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:50.353167 (+ 5033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:50.353219 (+ 52us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:50.353239 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:50.353282 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:50.356224 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:50.357109 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:50.357116 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:50.357123 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:50.357350 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:50.357799 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:50.357802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:50.357804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:50.358360 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:50.358597 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:50.359043 (+ 446us) server_negotiation.cc:299] Negotiation successful
0807 19:46:50.359425 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":310,"threads_started":1}
I20260807 19:46:50.364146 19316 raft_consensus.cc:348] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76b22f6b9fb4feda93a165a24d6f60f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 46491 } }
I20260807 19:46:50.364502 19316 raft_consensus.cc:374] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:50.364576 19316 raft_consensus.cc:729] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c76b22f6b9fb4feda93a165a24d6f60f, State: Initialized, Role: FOLLOWER
I20260807 19:46:50.365051 19316 consensus_queue.cc:260] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76b22f6b9fb4feda93a165a24d6f60f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 46491 } }
I20260807 19:46:50.365285 19316 raft_consensus.cc:388] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:46:50.365422 19316 raft_consensus.cc:482] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:46:50.365592 19316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:50.367966 19316 raft_consensus.cc:504] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76b22f6b9fb4feda93a165a24d6f60f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 46491 } }
I20260807 19:46:50.368427 19316 leader_election.cc:302] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c76b22f6b9fb4feda93a165a24d6f60f; no voters:
I20260807 19:46:50.369064 19316 leader_election.cc:288] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:46:50.369215 19323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:46:50.370930 19316 sys_catalog.cc:564] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:46:50.372112 19323 raft_consensus.cc:686] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [term 1 LEADER]: Becoming Leader. State: Replica: c76b22f6b9fb4feda93a165a24d6f60f, State: Running, Role: LEADER
I20260807 19:46:50.372779 19323 consensus_queue.cc:237] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76b22f6b9fb4feda93a165a24d6f60f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 46491 } }
I20260807 19:46:50.375735 19324 sys_catalog.cc:455] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c76b22f6b9fb4feda93a165a24d6f60f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76b22f6b9fb4feda93a165a24d6f60f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 46491 } } }
I20260807 19:46:50.375706 19325 sys_catalog.cc:455] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c76b22f6b9fb4feda93a165a24d6f60f. Latest consensus state: current_term: 1 leader_uuid: "c76b22f6b9fb4feda93a165a24d6f60f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c76b22f6b9fb4feda93a165a24d6f60f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 46491 } } }
I20260807 19:46:50.376109 19324 sys_catalog.cc:458] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:50.376128 19325 sys_catalog.cc:458] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:50.377774 19333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:46:50.382793 19333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:46:50.390838 19333 catalog_manager.cc:1347] Generated new cluster ID: 7ec27a95a8544e4d9e0e08e1b57df9a8
I20260807 19:46:50.390954 19333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:46:50.410825 19333 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:46:50.412114 19333 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:46:50.420511 19333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f: Generated new TSK 0
I20260807 19:46:50.421564 19333 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:46:50.500588 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:46491
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:40521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:50.838253 19344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:50.838549 19344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:50.838685 19344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:50.846092 19344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:50.846220 19344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:50.846375 19344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:46:50.856478 19344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:40521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:46491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:50.857880 19344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:50.859155 19344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:50.872437 19352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:50.872465 19349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:50.872490 19350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:50.873342 19344 server_base.cc:1034] running on GCE node
I20260807 19:46:50.873996 19344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:50.875058 19344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:50.876256 19344 hybrid_clock.cc:648] HybridClock initialized: now 1786132010876176 us; error 72 us; skew 500 ppm
I20260807 19:46:50.876575 19344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:50 dist-test-slave-vq2q krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:50.879675 19344 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:46:50.881116 19344 webserver.cc:460] Webserver started at http://127.30.208.1:38453/ using document root <none> and password file <none>
I20260807 19:46:50.881829 19344 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:50.881948 19344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:50.882246 19344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:50.885349 19344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "b1938920c48e46358210cb6cddf37968"
format_stamp: "Formatted at 2026-08-07 19:46:50 on dist-test-slave-vq2q"
server_key: "a9b8e5984aca80653269b5a066e19c14"
server_key_iv: "aa9712dcfa0280c460d247b8307e76b7"
server_key_version: "encryptionkey@0"
I20260807 19:46:50.886138 19344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "b1938920c48e46358210cb6cddf37968"
format_stamp: "Formatted at 2026-08-07 19:46:50 on dist-test-slave-vq2q"
server_key: "a9b8e5984aca80653269b5a066e19c14"
server_key_iv: "aa9712dcfa0280c460d247b8307e76b7"
server_key_version: "encryptionkey@0"
I20260807 19:46:50.891350 19344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:46:50.894739 19359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:50.896011 19344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:46:50.896167 19344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "b1938920c48e46358210cb6cddf37968"
format_stamp: "Formatted at 2026-08-07 19:46:50 on dist-test-slave-vq2q"
server_key: "a9b8e5984aca80653269b5a066e19c14"
server_key_iv: "aa9712dcfa0280c460d247b8307e76b7"
server_key_version: "encryptionkey@0"
I20260807 19:46:50.896376 19344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:50.934970 19344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:50.939393 19344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:50.939682 19344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:50.940667 19344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:50.942085 19344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:50.942167 19344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:50.942293 19344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:50.942348 19344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:50.973421 19344 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:41433
I20260807 19:46:50.973505 19481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:41433 every 8 connection(s)
I20260807 19:46:50.974711 19344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:46:50.980595 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 19344
I20260807 19:46:50.980803 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:46:50.981292 31552 external_mini_cluster.cc:1428] Setting key 8392cfb260e0aa4f18439f8a4ccbb63e
Aug 07 19:46:50 dist-test-slave-vq2q krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:50.996364 19485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:50.977159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:46205 (local address 127.30.208.62:46491)
0807 19:46:50.977643 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:50.977649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:50.978585 (+ 936us) server_negotiation.cc:407] Connection header received
0807 19:46:50.980390 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:50.980396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:50.980484 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:50.980605 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:50.982494 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:50.983492 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:50.985692 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:50.986101 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:50.986196 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:50.990174 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:50.990180 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:50.990184 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:50.990223 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:50.993251 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:50.994046 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:50.994052 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:50.994055 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:50.994195 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:50.994605 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:50.994609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:50.994612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:50.994941 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:50.995082 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:50.995860 (+ 778us) server_negotiation.cc:299] Negotiation successful
0807 19:46:50.996090 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":218,"threads_started":1}
I20260807 19:46:50.997684 19486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:50.977731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:46491 (local address 127.30.208.1:46205)
0807 19:46:50.978422 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:50.978468 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:50.980133 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:50.980841 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:50.980853 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:50.981495 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:50.982306 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:50.982335 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:50.983657 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:50.983662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:50.985555 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:50.985571 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:50.986314 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:50.986320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:50.986578 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:50.987349 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:50.987377 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:50.989925 (+ 2548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:50.993390 (+ 3465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:50.993398 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:50.993414 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:50.993913 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:50.994286 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:50.994289 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:50.994292 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:50.994502 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:50.995068 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:50.995075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:50.995457 (+ 382us) client_negotiation.cc:769] Sending connection context
0807 19:46:50.995710 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 19:46:50.996217 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":267,"threads_started":1}
I20260807 19:46:50.999372 19482 heartbeater.cc:347] Connected to a master server at 127.30.208.62:46491
I20260807 19:46:50.999797 19482 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:51.000623 19482 heartbeater.cc:511] Master 127.30.208.62:46491 requested a full tablet report, sending...
I20260807 19:46:51.002862 19276 ts_manager.cc:194] Registered new tserver with Master: b1938920c48e46358210cb6cddf37968 (127.30.208.1:41433)
I20260807 19:46:51.004420 19276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:46205
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:46:51.049119 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:46491
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:40521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:51.379428 19489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:51.379698 19489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:51.379832 19489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:51.387094 19489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:51.387207 19489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:51.387372 19489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:46:51.397560 19489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:40521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:46491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:51.398866 19489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:51.400214 19489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:51.413086 19495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:51.413053 19498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:51.413044 19496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:51.413885 19489 server_base.cc:1034] running on GCE node
I20260807 19:46:51.414453 19489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:51.415345 19489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:51.416548 19489 hybrid_clock.cc:648] HybridClock initialized: now 1786132011416500 us; error 33 us; skew 500 ppm
I20260807 19:46:51.416850 19489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:51 dist-test-slave-vq2q krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:51.424769 19489 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:46:51.426306 19489 webserver.cc:460] Webserver started at http://127.30.208.2:37819/ using document root <none> and password file <none>
I20260807 19:46:51.427024 19489 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:51.427150 19489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:51.427476 19489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:51.430511 19489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "85285ca3d2864b4ca0a2d8edee55d9d3"
format_stamp: "Formatted at 2026-08-07 19:46:51 on dist-test-slave-vq2q"
server_key: "18cd9deae2ab640d93b236c562eedeb4"
server_key_iv: "df7e8ab3430ad23e55a03daeaa3c9c6f"
server_key_version: "encryptionkey@0"
I20260807 19:46:51.431260 19489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "85285ca3d2864b4ca0a2d8edee55d9d3"
format_stamp: "Formatted at 2026-08-07 19:46:51 on dist-test-slave-vq2q"
server_key: "18cd9deae2ab640d93b236c562eedeb4"
server_key_iv: "df7e8ab3430ad23e55a03daeaa3c9c6f"
server_key_version: "encryptionkey@0"
I20260807 19:46:51.436486 19489 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260807 19:46:51.439885 19507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:51.441112 19489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:46:51.441294 19489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "85285ca3d2864b4ca0a2d8edee55d9d3"
format_stamp: "Formatted at 2026-08-07 19:46:51 on dist-test-slave-vq2q"
server_key: "18cd9deae2ab640d93b236c562eedeb4"
server_key_iv: "df7e8ab3430ad23e55a03daeaa3c9c6f"
server_key_version: "encryptionkey@0"
I20260807 19:46:51.441485 19489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:51.466930 19489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:51.470857 19489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:51.471136 19489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:51.472088 19489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:51.473423 19489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:51.473501 19489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:51.473623 19489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:51.473677 19489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:51.503568 19489 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:44899
I20260807 19:46:51.503615 19628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:44899 every 8 connection(s)
I20260807 19:46:51.505123 19489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:46:51.507951 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 19489
I20260807 19:46:51.508144 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:46:51.508819 31552 external_mini_cluster.cc:1428] Setting key 32e7b7c0c8814e27b9981cef48c4f49e
Aug 07 19:46:51 dist-test-slave-vq2q krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:51.526520 19632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:51.507645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:33231 (local address 127.30.208.62:46491)
0807 19:46:51.510773 (+ 3128us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:51.510779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:51.510800 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:46:51.512168 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:51.512174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:51.512252 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:51.512346 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:51.513894 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:51.514614 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:51.516128 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:51.516562 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:51.516672 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:51.520253 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:51.520259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:51.520262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:51.520296 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:51.523393 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:51.524131 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:51.524135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:51.524138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:51.524278 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:51.524714 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:51.524718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:51.524720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:51.525022 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:51.525159 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:51.525909 (+ 750us) server_negotiation.cc:299] Negotiation successful
0807 19:46:51.526144 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3011,"thread_start_us":2896,"threads_started":1}
I20260807 19:46:51.527494 19633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:51.508261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:46491 (local address 127.30.208.2:33231)
0807 19:46:51.510049 (+ 1788us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:51.510098 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:51.511910 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:51.512592 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:51.512600 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:51.513118 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:51.513720 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:51.513738 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:51.514760 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:51.514764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:51.515977 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:51.515989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:51.516779 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:51.516786 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:51.517013 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:51.517727 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:51.517751 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:51.520014 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:51.523528 (+ 3514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:51.523538 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:51.523556 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:51.523995 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:51.524388 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:51.524393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:51.524396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:51.524596 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:51.525154 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:51.525162 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:51.525528 (+ 366us) client_negotiation.cc:769] Sending connection context
0807 19:46:51.525778 (+ 250us) client_negotiation.cc:240] Negotiation successful
0807 19:46:51.526334 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1507,"thread_start_us":1266,"threads_started":1}
I20260807 19:46:51.528910 19629 heartbeater.cc:347] Connected to a master server at 127.30.208.62:46491
I20260807 19:46:51.529282 19629 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:51.530164 19629 heartbeater.cc:511] Master 127.30.208.62:46491 requested a full tablet report, sending...
I20260807 19:46:51.531811 19276 ts_manager.cc:194] Registered new tserver with Master: 85285ca3d2864b4ca0a2d8edee55d9d3 (127.30.208.2:44899)
I20260807 19:46:51.532688 19276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:33231
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:46:51.574429 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:46491
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:40521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:51.915274 19637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:51.915572 19637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:51.915700 19637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:51.923128 19637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:51.923250 19637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:51.923401 19637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:46:51.933318 19637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:40521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:46491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:51.934626 19637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:51.935940 19637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:51.948769 19646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:51.948796 19644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:51.948819 19642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:51.949514 19637 server_base.cc:1034] running on GCE node
I20260807 19:46:51.950115 19637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:51.951057 19637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:51.952267 19637 hybrid_clock.cc:648] HybridClock initialized: now 1786132011952214 us; error 41 us; skew 500 ppm
I20260807 19:46:51.952571 19637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:51 dist-test-slave-vq2q krb5kdc[19228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:51.955632 19637 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:46:51.957150 19637 webserver.cc:460] Webserver started at http://127.30.208.3:43585/ using document root <none> and password file <none>
I20260807 19:46:51.957866 19637 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:51.957978 19637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:51.958281 19637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:51.961112 19637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "42a1bcee9cb4416abb2abf668a97d8f6"
format_stamp: "Formatted at 2026-08-07 19:46:51 on dist-test-slave-vq2q"
server_key: "ae9c479c93db6f6811d8a3258b9a448a"
server_key_iv: "741cb44e08e197948a517b312f2d55a7"
server_key_version: "encryptionkey@0"
I20260807 19:46:51.961890 19637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "42a1bcee9cb4416abb2abf668a97d8f6"
format_stamp: "Formatted at 2026-08-07 19:46:51 on dist-test-slave-vq2q"
server_key: "ae9c479c93db6f6811d8a3258b9a448a"
server_key_iv: "741cb44e08e197948a517b312f2d55a7"
server_key_version: "encryptionkey@0"
I20260807 19:46:51.966985 19637 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:46:51.970314 19655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:51.971788 19637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:46:51.971949 19637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "42a1bcee9cb4416abb2abf668a97d8f6"
format_stamp: "Formatted at 2026-08-07 19:46:51 on dist-test-slave-vq2q"
server_key: "ae9c479c93db6f6811d8a3258b9a448a"
server_key_iv: "741cb44e08e197948a517b312f2d55a7"
server_key_version: "encryptionkey@0"
I20260807 19:46:51.972144 19637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:51.985966 19637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:51.989944 19637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:51.990213 19637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:51.991195 19637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:51.992565 19637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:51.992643 19637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:51.992770 19637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:51.992825 19637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:52.007804 19482 heartbeater.cc:503] Master 127.30.208.62:46491 was elected leader, sending a full tablet report...
I20260807 19:46:52.025197 19637 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:40505
I20260807 19:46:52.025264 19777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:40505 every 8 connection(s)
I20260807 19:46:52.026556 19637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:46:52.027114 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 19637
I20260807 19:46:52.027354 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:46:52.027946 31552 external_mini_cluster.cc:1428] Setting key 84b66db6b9f145423bf2890fa1b06ea0
Aug 07 19:46:52 dist-test-slave-vq2q krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132011, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:52.047112 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.029419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:43133 (local address 127.30.208.62:46491)
0807 19:46:52.029856 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:52.029862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:52.031215 (+ 1353us) server_negotiation.cc:407] Connection header received
0807 19:46:52.033045 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:52.033049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:52.033129 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:52.033257 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:52.034717 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.035509 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.036959 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.037406 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.037500 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.041023 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:52.041029 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:52.041032 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:52.041068 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:52.044108 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:52.044867 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:52.044873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:52.044877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:52.045031 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:52.045492 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:52.045496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:52.045498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:52.045777 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:52.045928 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:52.046609 (+ 681us) server_negotiation.cc:299] Negotiation successful
0807 19:46:52.046796 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":199,"threads_started":1}
I20260807 19:46:52.048089 19781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.030113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:46491 (local address 127.30.208.3:43133)
0807 19:46:52.031032 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:52.031093 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:52.032798 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:52.033435 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:52.033443 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:52.033999 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:52.034556 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:52.034573 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.035620 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.035624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:52.036835 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:52.036848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.037628 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.037634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:52.037877 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.038526 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:52.038549 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:52.040799 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:52.044241 (+ 3442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:52.044249 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:52.044267 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:52.044708 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:52.045142 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:52.045146 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:52.045148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:52.045401 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:52.045900 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:52.045906 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:52.046285 (+ 379us) client_negotiation.cc:769] Sending connection context
0807 19:46:52.046522 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 19:46:52.046985 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":330,"threads_started":1}
I20260807 19:46:52.049449 19778 heartbeater.cc:347] Connected to a master server at 127.30.208.62:46491
I20260807 19:46:52.049732 19778 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:52.050441 19778 heartbeater.cc:511] Master 127.30.208.62:46491 requested a full tablet report, sending...
I20260807 19:46:52.051824 19276 ts_manager.cc:194] Registered new tserver with Master: 42a1bcee9cb4416abb2abf668a97d8f6 (127.30.208.3:40505)
I20260807 19:46:52.052702 19276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:43133
I20260807 19:46:52.057011 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:46:52.074627 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.061066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57264 (local address 127.30.208.62:46491)
0807 19:46:52.061256 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:52.061262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:52.061684 (+ 422us) server_negotiation.cc:407] Connection header received
0807 19:46:52.061846 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:52.061851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:52.061924 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:52.062004 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:52.063467 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.064188 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.065741 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.066202 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.066327 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.069339 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:52.069345 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:52.069348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:52.069391 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:52.071964 (+ 2573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:52.072712 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:52.072717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:52.072719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:52.072874 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:52.073304 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:52.073308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:52.073310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:52.073619 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:52.073811 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:52.074138 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 19:46:52.074310 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 19:46:52.078773 19276 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:46:52.085824 19276 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:46:52.121367 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.101101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49608 (local address 127.30.208.3:40505)
0807 19:46:52.112695 (+ 11594us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:52.112700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:52.112730 (+ 30us) server_negotiation.cc:407] Connection header received
0807 19:46:52.112806 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:52.112810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:52.112977 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:52.113244 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:52.113817 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.115297 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.118564 (+ 3267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.119953 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.120078 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.120319 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:52.120840 (+ 521us) server_negotiation.cc:299] Negotiation successful
0807 19:46:52.121008 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11452,"thread_start_us":11293,"threads_started":1}
I20260807 19:46:52.121399 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.100710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:40505 (local address 127.0.0.1:49608)
0807 19:46:52.111957 (+ 11247us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:52.111979 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:52.112121 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:52.113236 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:52.113240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:52.113273 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:52.113637 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:52.113649 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.115584 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.115589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:52.118363 (+ 2774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:52.118383 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.120438 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.120443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:52.120585 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.120620 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:46:52.120737 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 19:46:52.120940 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11120,"mutex_wait_us":249,"thread_start_us":9909,"threads_started":1}
I20260807 19:46:52.122704 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.099223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49790 (local address 127.30.208.1:41433)
0807 19:46:52.110566 (+ 11343us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:52.110576 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:52.111659 (+ 1083us) server_negotiation.cc:407] Connection header received
0807 19:46:52.111814 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:52.111823 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:52.112105 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:52.112331 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:52.113355 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.114613 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.119673 (+ 5060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.121210 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.121356 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.121583 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:52.122178 (+ 595us) server_negotiation.cc:299] Negotiation successful
0807 19:46:52.122348 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11160,"thread_start_us":10987,"threads_started":1}
I20260807 19:46:52.122720 19799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.098643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:41433 (local address 127.0.0.1:49790)
0807 19:46:52.110878 (+ 12235us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:52.110897 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:52.111055 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:52.112609 (+ 1554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:52.112613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:52.112636 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:52.113121 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:52.113146 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.114785 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.114790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:52.119433 (+ 4643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:52.119452 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.121750 (+ 2298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.121757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:52.121951 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.121989 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 19:46:52.122064 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 19:46:52.122223 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12109,"mutex_wait_us":601,"thread_start_us":761,"threads_started":1}
I20260807 19:46:52.122756 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.098487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:44899 (local address 127.0.0.1:38908)
0807 19:46:52.099609 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:52.099657 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:52.099825 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:52.112541 (+ 12716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:52.112549 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:52.112581 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:52.113273 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:52.113291 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.115102 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.115107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:52.119433 (+ 4326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:52.119452 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.121892 (+ 2440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.121900 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:52.122070 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.122102 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:46:52.122193 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 19:46:52.122346 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":470,"thread_start_us":13183,"threads_started":1}
I20260807 19:46:52.125113 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.098727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38908 (local address 127.30.208.2:44899)
0807 19:46:52.111951 (+ 13224us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:52.111960 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:52.111987 (+ 27us) server_negotiation.cc:407] Connection header received
0807 19:46:52.112073 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:52.112079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:52.112294 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:52.112952 (+ 658us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:52.113512 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.114962 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.119682 (+ 4720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.121210 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.121356 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.121913 (+ 557us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:52.124487 (+ 2574us) server_negotiation.cc:299] Negotiation successful
0807 19:46:52.124676 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12679,"thread_start_us":12568,"threads_started":1}
I20260807 19:46:52.126047 19709 tablet_service.cc:1467] Processing CreateTablet for tablet cf065ffc7fa244059aa678dec9a71d35 (DEFAULT_TABLE table=test-table [id=82cd50a8cc314b4c84968cee4b9baa5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:52.126036 19414 tablet_service.cc:1467] Processing CreateTablet for tablet cf065ffc7fa244059aa678dec9a71d35 (DEFAULT_TABLE table=test-table [id=82cd50a8cc314b4c84968cee4b9baa5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:52.127420 19709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf065ffc7fa244059aa678dec9a71d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:52.127517 19414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf065ffc7fa244059aa678dec9a71d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:52.128988 19560 tablet_service.cc:1467] Processing CreateTablet for tablet cf065ffc7fa244059aa678dec9a71d35 (DEFAULT_TABLE table=test-table [id=82cd50a8cc314b4c84968cee4b9baa5a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:52.130771 19560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf065ffc7fa244059aa678dec9a71d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:52.139238 19801 tablet_bootstrap.cc:492] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6: Bootstrap starting.
I20260807 19:46:52.141633 19802 tablet_bootstrap.cc:492] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968: Bootstrap starting.
I20260807 19:46:52.143040 19801 tablet_bootstrap.cc:654] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:52.144582 19801 log.cc:824] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:52.145263 19802 tablet_bootstrap.cc:654] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:52.146287 19804 tablet_bootstrap.cc:492] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3: Bootstrap starting.
I20260807 19:46:52.146577 19802 log.cc:824] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:52.148422 19801 tablet_bootstrap.cc:492] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6: No bootstrap required, opened a new log
I20260807 19:46:52.148717 19801 ts_tablet_manager.cc:1397] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 19:46:52.149806 19802 tablet_bootstrap.cc:492] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968: No bootstrap required, opened a new log
I20260807 19:46:52.150055 19802 ts_tablet_manager.cc:1397] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260807 19:46:52.151041 19804 tablet_bootstrap.cc:654] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:52.152236 19804 log.cc:824] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:52.155352 19804 tablet_bootstrap.cc:492] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3: No bootstrap required, opened a new log
I20260807 19:46:52.155619 19804 ts_tablet_manager.cc:1397] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260807 19:46:52.157320 19802 raft_consensus.cc:348] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85285ca3d2864b4ca0a2d8edee55d9d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44899 } } peers { permanent_uuid: "b1938920c48e46358210cb6cddf37968" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41433 } } peers { permanent_uuid: "42a1bcee9cb4416abb2abf668a97d8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40505 } }
I20260807 19:46:52.157660 19802 raft_consensus.cc:374] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:52.157758 19802 raft_consensus.cc:729] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1938920c48e46358210cb6cddf37968, State: Initialized, Role: FOLLOWER
I20260807 19:46:52.158377 19802 consensus_queue.cc:260] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85285ca3d2864b4ca0a2d8edee55d9d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44899 } } peers { permanent_uuid: "b1938920c48e46358210cb6cddf37968" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41433 } } peers { permanent_uuid: "42a1bcee9cb4416abb2abf668a97d8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40505 } }
I20260807 19:46:52.158601 19801 raft_consensus.cc:348] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85285ca3d2864b4ca0a2d8edee55d9d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44899 } } peers { permanent_uuid: "b1938920c48e46358210cb6cddf37968" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41433 } } peers { permanent_uuid: "42a1bcee9cb4416abb2abf668a97d8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40505 } }
I20260807 19:46:52.158998 19801 raft_consensus.cc:374] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:52.159076 19801 raft_consensus.cc:729] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a1bcee9cb4416abb2abf668a97d8f6, State: Initialized, Role: FOLLOWER
I20260807 19:46:52.159745 19801 consensus_queue.cc:260] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85285ca3d2864b4ca0a2d8edee55d9d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44899 } } peers { permanent_uuid: "b1938920c48e46358210cb6cddf37968" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41433 } } peers { permanent_uuid: "42a1bcee9cb4416abb2abf668a97d8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40505 } }
I20260807 19:46:52.161623 19802 ts_tablet_manager.cc:1428] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260807 19:46:52.166487 19804 raft_consensus.cc:348] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85285ca3d2864b4ca0a2d8edee55d9d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44899 } } peers { permanent_uuid: "b1938920c48e46358210cb6cddf37968" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41433 } } peers { permanent_uuid: "42a1bcee9cb4416abb2abf668a97d8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40505 } }
I20260807 19:46:52.166813 19778 heartbeater.cc:503] Master 127.30.208.62:46491 was elected leader, sending a full tablet report...
I20260807 19:46:52.166977 19804 raft_consensus.cc:374] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:52.167080 19804 raft_consensus.cc:729] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85285ca3d2864b4ca0a2d8edee55d9d3, State: Initialized, Role: FOLLOWER
I20260807 19:46:52.167888 19801 ts_tablet_manager.cc:1428] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6: Time spent starting tablet: real 0.019s user 0.015s sys 0.004s
I20260807 19:46:52.167721 19804 consensus_queue.cc:260] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85285ca3d2864b4ca0a2d8edee55d9d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44899 } } peers { permanent_uuid: "b1938920c48e46358210cb6cddf37968" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41433 } } peers { permanent_uuid: "42a1bcee9cb4416abb2abf668a97d8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40505 } }
I20260807 19:46:52.168783 19629 heartbeater.cc:503] Master 127.30.208.62:46491 was elected leader, sending a full tablet report...
I20260807 19:46:52.169479 19804 ts_tablet_manager.cc:1428] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260807 19:46:52.226150 19807 raft_consensus.cc:482] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:52.226502 19807 raft_consensus.cc:504] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85285ca3d2864b4ca0a2d8edee55d9d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44899 } } peers { permanent_uuid: "b1938920c48e46358210cb6cddf37968" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41433 } } peers { permanent_uuid: "42a1bcee9cb4416abb2abf668a97d8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40505 } }
W20260807 19:46:52.227165 19484 tablet.cc:2369] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:46:52.228358 19807 leader_election.cc:288] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85285ca3d2864b4ca0a2d8edee55d9d3 (127.30.208.2:44899), 42a1bcee9cb4416abb2abf668a97d8f6 (127.30.208.3:40505)
I20260807 19:46:52.237689 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.228949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:39433 (local address 127.30.208.3:40505)
0807 19:46:52.229151 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:52.229157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:52.229929 (+ 772us) server_negotiation.cc:407] Connection header received
0807 19:46:52.230141 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:52.230147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:52.230230 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:52.230341 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:52.231218 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.232336 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.235332 (+ 2996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.236550 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.236720 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.236793 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:52.237299 (+ 506us) server_negotiation.cc:299] Negotiation successful
0807 19:46:52.237429 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:46:52.237687 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.229032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:36941 (local address 127.30.208.2:44899)
0807 19:46:52.229265 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:52.229271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:52.229995 (+ 724us) server_negotiation.cc:407] Connection header received
0807 19:46:52.230155 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:52.230159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:52.230230 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:52.230349 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:52.231231 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.232329 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.235332 (+ 3003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.236572 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.236715 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.236793 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:52.237279 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 19:46:52.237429 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:46:52.237869 19811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.228709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:40505 (local address 127.30.208.1:39433)
0807 19:46:52.229789 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:52.229810 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:52.229989 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:52.230398 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:52.230404 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:52.230439 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:52.231048 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:52.231067 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.232506 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.232511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:52.235147 (+ 2636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:52.235158 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.236812 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.236819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:52.237034 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.237123 (+ 89us) client_negotiation.cc:769] Sending connection context
0807 19:46:52.237263 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 19:46:52.237395 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"mutex_wait_us":199,"thread_start_us":787,"threads_started":1}
I20260807 19:46:52.238102 19810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.228854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:44899 (local address 127.30.208.1:36941)
0807 19:46:52.229870 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:52.229890 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:52.230041 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:52.230398 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:52.230404 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:52.230451 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:52.231076 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:52.231086 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.232503 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.232509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:52.235141 (+ 2632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:52.235158 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.236853 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.236859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:52.237074 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.237125 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 19:46:52.237242 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 19:46:52.237395 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":918,"mutex_wait_us":208,"spinlock_wait_cycles":62080,"thread_start_us":480,"threads_started":1}
I20260807 19:46:52.239024 19729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf065ffc7fa244059aa678dec9a71d35" candidate_uuid: "b1938920c48e46358210cb6cddf37968" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42a1bcee9cb4416abb2abf668a97d8f6" is_pre_election: true
I20260807 19:46:52.239125 19581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf065ffc7fa244059aa678dec9a71d35" candidate_uuid: "b1938920c48e46358210cb6cddf37968" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85285ca3d2864b4ca0a2d8edee55d9d3" is_pre_election: true
I20260807 19:46:52.239675 19581 raft_consensus.cc:2454] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1938920c48e46358210cb6cddf37968 in term 0.
I20260807 19:46:52.239674 19729 raft_consensus.cc:2454] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1938920c48e46358210cb6cddf37968 in term 0.
I20260807 19:46:52.240450 19364 leader_election.cc:302] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85285ca3d2864b4ca0a2d8edee55d9d3, b1938920c48e46358210cb6cddf37968; no voters:
I20260807 19:46:52.240903 19807 raft_consensus.cc:2781] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:46:52.241070 19807 raft_consensus.cc:482] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:46:52.241163 19807 raft_consensus.cc:3037] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:52.243474 19807 raft_consensus.cc:504] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85285ca3d2864b4ca0a2d8edee55d9d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44899 } } peers { permanent_uuid: "b1938920c48e46358210cb6cddf37968" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41433 } } peers { permanent_uuid: "42a1bcee9cb4416abb2abf668a97d8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40505 } }
I20260807 19:46:52.244052 19807 leader_election.cc:288] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [CANDIDATE]: Term 1 election: Requested vote from peers 85285ca3d2864b4ca0a2d8edee55d9d3 (127.30.208.2:44899), 42a1bcee9cb4416abb2abf668a97d8f6 (127.30.208.3:40505)
I20260807 19:46:52.244554 19581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf065ffc7fa244059aa678dec9a71d35" candidate_uuid: "b1938920c48e46358210cb6cddf37968" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85285ca3d2864b4ca0a2d8edee55d9d3"
I20260807 19:46:52.244620 19729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf065ffc7fa244059aa678dec9a71d35" candidate_uuid: "b1938920c48e46358210cb6cddf37968" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42a1bcee9cb4416abb2abf668a97d8f6"
I20260807 19:46:52.244742 19581 raft_consensus.cc:3037] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:52.244836 19729 raft_consensus.cc:3037] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:52.247421 19729 raft_consensus.cc:2454] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1938920c48e46358210cb6cddf37968 in term 1.
I20260807 19:46:52.247480 19581 raft_consensus.cc:2454] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1938920c48e46358210cb6cddf37968 in term 1.
I20260807 19:46:52.247892 19361 leader_election.cc:302] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42a1bcee9cb4416abb2abf668a97d8f6, b1938920c48e46358210cb6cddf37968; no voters:
I20260807 19:46:52.248220 19807 raft_consensus.cc:2781] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:46:52.248664 19807 raft_consensus.cc:686] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [term 1 LEADER]: Becoming Leader. State: Replica: b1938920c48e46358210cb6cddf37968, State: Running, Role: LEADER
I20260807 19:46:52.249306 19807 consensus_queue.cc:237] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85285ca3d2864b4ca0a2d8edee55d9d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44899 } } peers { permanent_uuid: "b1938920c48e46358210cb6cddf37968" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41433 } } peers { permanent_uuid: "42a1bcee9cb4416abb2abf668a97d8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40505 } }
I20260807 19:46:52.254170 19274 catalog_manager.cc:5495] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1938920c48e46358210cb6cddf37968 (127.30.208.1). New cstate: current_term: 1 leader_uuid: "b1938920c48e46358210cb6cddf37968" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85285ca3d2864b4ca0a2d8edee55d9d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1938920c48e46358210cb6cddf37968" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 41433 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42a1bcee9cb4416abb2abf668a97d8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40505 } health_report { overall_health: UNKNOWN } } }
W20260807 19:46:52.257141 19630 tablet.cc:2369] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:46:52.276453 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.270735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49792 (local address 127.30.208.1:41433)
0807 19:46:52.270948 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:52.270954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:52.270973 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:46:52.271167 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:52.271172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:52.271278 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:52.271449 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:46:52.271849 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.272608 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.274113 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.274519 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.274631 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.275222 (+ 591us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:46:52.275810 (+ 588us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:46:52.275981 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:52.276134 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 19:46:52.276183 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 19:46:52.278258 19779 tablet.cc:2369] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:46:52.284811 19729 raft_consensus.cc:1261] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6 [term 1 FOLLOWER]: Refusing update from remote peer b1938920c48e46358210cb6cddf37968: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:46:52.284816 19581 raft_consensus.cc:1261] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3 [term 1 FOLLOWER]: Refusing update from remote peer b1938920c48e46358210cb6cddf37968: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:46:52.285790 19812 consensus_queue.cc:1035] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42a1bcee9cb4416abb2abf668a97d8f6" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 40505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:46:52.286144 19807 consensus_queue.cc:1035] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85285ca3d2864b4ca0a2d8edee55d9d3" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:46:52.299147 19816 mvcc.cc:205] Tried to move back new op lower bound from 7315996722308734976 to 7315996722177318912. Current Snapshot: MvccSnapshot[applied={T|T < 7315996722308734976}]
I20260807 19:46:52.314530 19276 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:46:52.314859 19276 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:46:52.319264 19276 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c76b22f6b9fb4feda93a165a24d6f60f: Sending DeleteTablet for 3 replicas of tablet cf065ffc7fa244059aa678dec9a71d35
I20260807 19:46:52.320528 19818 mvcc.cc:205] Tried to move back new op lower bound from 7315996722308734976 to 7315996722177318912. Current Snapshot: MvccSnapshot[applied={T|T < 7315996722308734976}]
I20260807 19:46:52.321085 19709 tablet_service.cc:1514] Processing DeleteTablet for tablet cf065ffc7fa244059aa678dec9a71d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:52 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:49608
I20260807 19:46:52.322098 19414 tablet_service.cc:1514] Processing DeleteTablet for tablet cf065ffc7fa244059aa678dec9a71d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:52 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:49790
I20260807 19:46:52.322865 19560 tablet_service.cc:1514] Processing DeleteTablet for tablet cf065ffc7fa244059aa678dec9a71d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:52 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:38908
I20260807 19:46:52.323732 19829 tablet_replica.cc:330] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3: stopping tablet replica
I20260807 19:46:52.324205 19829 raft_consensus.cc:2229] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:46:52.324745 19829 raft_consensus.cc:2258] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:52.325691 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 19344
I20260807 19:46:52.330928 19829 ts_tablet_manager.cc:1905] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:52.335319 19827 tablet_replica.cc:330] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6: stopping tablet replica
I20260807 19:46:52.335815 19827 raft_consensus.cc:2229] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:46:52.336359 19827 raft_consensus.cc:2258] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 07 19:46:52 dist-test-slave-vq2q krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:46:52.338651 19827 ts_tablet_manager.cc:1905] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:52.341418 19829 ts_tablet_manager.cc:1918] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:52.341890 19829 log.cc:1196] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/cf065ffc7fa244059aa678dec9a71d35
I20260807 19:46:52.342495 19829 ts_tablet_manager.cc:1939] T cf065ffc7fa244059aa678dec9a71d35 P 85285ca3d2864b4ca0a2d8edee55d9d3: Deleting consensus metadata
I20260807 19:46:52.344409 19262 catalog_manager.cc:4841] TS 85285ca3d2864b4ca0a2d8edee55d9d3 (127.30.208.2:44899): tablet cf065ffc7fa244059aa678dec9a71d35 (table test-table [id=82cd50a8cc314b4c84968cee4b9baa5a]) successfully deleted
I20260807 19:46:52.346228 19827 ts_tablet_manager.cc:1918] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:52.346465 19827 log.cc:1196] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/cf065ffc7fa244059aa678dec9a71d35
I20260807 19:46:52.347071 19827 ts_tablet_manager.cc:1939] T cf065ffc7fa244059aa678dec9a71d35 P 42a1bcee9cb4416abb2abf668a97d8f6: Deleting consensus metadata
I20260807 19:46:52.347654 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.326836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.30.208.1:41433)
0807 19:46:52.327029 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:52.327035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:52.327418 (+ 383us) server_negotiation.cc:407] Connection header received
0807 19:46:52.327684 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:52.327689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:52.327774 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:52.327846 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:52.329751 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.330890 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.332935 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.333709 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.333816 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.339676 (+ 5860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:52.339695 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:52.339713 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:52.339764 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:52.343648 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:52.344623 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:52.344632 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:52.344636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:52.344914 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:52.345531 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:52.345537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:52.345541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:52.346077 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:52.346357 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:52.346993 (+ 636us) server_negotiation.cc:299] Negotiation successful
0807 19:46:52.347255 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 19:46:52.349117 19475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:52.349193 19260 catalog_manager.cc:4841] TS 42a1bcee9cb4416abb2abf668a97d8f6 (127.30.208.3:40505): tablet cf065ffc7fa244059aa678dec9a71d35 (table test-table [id=82cd50a8cc314b4c84968cee4b9baa5a]) successfully deleted
I20260807 19:46:52.451882 19828 tablet_replica.cc:330] T cf065ffc7fa244059aa678dec9a71d35 P b1938920c48e46358210cb6cddf37968: stopping tablet replica
I20260807 19:46:52.452145 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 19344
I20260807 19:46:52.470572 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 19489
W20260807 19:46:52.470734 19261 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.208.1:41433 (error 108)
I20260807 19:46:52.472674 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.472225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:41433 (local address 127.0.0.1:49812)
0807 19:46:52.472440 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:52.472534 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 19:46:52.472875 19261 catalog_manager.cc:4584] TS b1938920c48e46358210cb6cddf37968 (127.30.208.1:41433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf065ffc7fa244059aa678dec9a71d35: Network error: Client connection negotiation failed: client connection to 127.30.208.1:41433: connect: Connection refused (error 111)
Aug 07 19:46:52 dist-test-slave-vq2q krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:46:52.486454 19797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.471358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38910 (local address 127.30.208.2:44899)
0807 19:46:52.471753 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:52.471758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:52.471775 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:46:52.471848 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:52.471853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:52.471921 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:52.472000 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:52.473394 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.474187 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.475600 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.476100 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.476224 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.480356 (+ 4132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:52.480371 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:52.480386 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:52.480426 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:52.483352 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:52.484164 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:52.484172 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:52.484175 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:52.484316 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:52.484779 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:52.484783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:52.484785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:52.485169 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:52.485519 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:52.485943 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 19:46:52.486120 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262}
I20260807 19:46:52.487124 19624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:52.526715 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.526206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:41433 (local address 127.0.0.1:49822)
0807 19:46:52.526439 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:52.526543 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 19:46:52.527793 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.527441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:41433 (local address 127.0.0.1:49826)
0807 19:46:52.527595 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:52.527663 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 19:46:52.577692 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 19489
I20260807 19:46:52.588315 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.587683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:41433 (local address 127.0.0.1:49828)
0807 19:46:52.587981 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:52.588102 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 19:46:52.589380 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.589010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:41433 (local address 127.0.0.1:49840)
0807 19:46:52.589150 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:52.589258 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 19:46:52.595835 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 19637
Aug 07 19:46:52 dist-test-slave-vq2q krb5kdc[19228](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:46:52.611302 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.596553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.30.208.3:40505)
0807 19:46:52.596736 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:52.596741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:52.596772 (+ 31us) server_negotiation.cc:407] Connection header received
0807 19:46:52.596885 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:52.596889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:52.596978 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:52.597065 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:52.598431 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.599183 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.600499 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:52.600903 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:52.601015 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:52.605908 (+ 4893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:52.605925 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:52.605942 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:52.605982 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:52.608673 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:52.609376 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:52.609383 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:52.609385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:52.609533 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:52.609890 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:52.609894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:52.609896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:52.610285 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:52.610521 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:52.610801 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 19:46:52.610980 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 19:46:52.611987 19773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:52.670717 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.669546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:41433 (local address 127.0.0.1:49852)
0807 19:46:52.669962 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:52.670089 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 19:46:52.672690 19796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:52.672108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:41433 (local address 127.0.0.1:49860)
0807 19:46:52.672392 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:52.672498 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260807 19:46:52.704728 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 19637
I20260807 19:46:52.722009 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 19242
I20260807 19:46:52.723157 19310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:52.835929 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 19242
2026-08-07T19:46:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4336 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T19:46:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:46:52Z Disabled control of system clock
I20260807 19:46:52.925833 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:37759
--webserver_interface=127.30.208.62
--webserver_port=0
--builtin_ntp_servers=127.30.208.20:45159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:37759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:46:53.263696 19840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:53.263975 19840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:53.264068 19840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:53.271130 19840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:46:53.271250 19840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:53.271299 19840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:53.271363 19840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:46:53.271412 19840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:46:53.280576 19840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:37759
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:37759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:53.281828 19840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:53.283550 19840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:53.295033 19846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:53.295010 19847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:53.294982 19850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:53.296028 19840 server_base.cc:1034] running on GCE node
I20260807 19:46:53.296842 19840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:53.298429 19840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:53.299645 19840 hybrid_clock.cc:648] HybridClock initialized: now 1786132013299587 us; error 53 us; skew 500 ppm
I20260807 19:46:53.300024 19840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:46:53.302306 19840 webserver.cc:460] Webserver started at http://127.30.208.62:39271/ using document root <none> and password file <none>
I20260807 19:46:53.303035 19840 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:53.303138 19840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:53.303474 19840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:53.306452 19840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "573d4af524ef4122a054be3646c40a5c"
format_stamp: "Formatted at 2026-08-07 19:46:53 on dist-test-slave-vq2q"
server_key: "af23827a87801bcbd97d8326a75456f5"
server_key_iv: "0a6b089d1a8022ce05d46655807ab278"
server_key_version: "encryptionkey@0"
I20260807 19:46:53.307256 19840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "573d4af524ef4122a054be3646c40a5c"
format_stamp: "Formatted at 2026-08-07 19:46:53 on dist-test-slave-vq2q"
server_key: "af23827a87801bcbd97d8326a75456f5"
server_key_iv: "0a6b089d1a8022ce05d46655807ab278"
server_key_version: "encryptionkey@0"
I20260807 19:46:53.312382 19840 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 19:46:53.315845 19856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:53.317134 19840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:46:53.317370 19840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "573d4af524ef4122a054be3646c40a5c"
format_stamp: "Formatted at 2026-08-07 19:46:53 on dist-test-slave-vq2q"
server_key: "af23827a87801bcbd97d8326a75456f5"
server_key_iv: "0a6b089d1a8022ce05d46655807ab278"
server_key_version: "encryptionkey@0"
I20260807 19:46:53.317582 19840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:53.334832 19840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:53.335628 19840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:53.335894 19840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:53.352993 19840 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:37759
I20260807 19:46:53.353024 19910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:37759 every 8 connection(s)
I20260807 19:46:53.354296 19840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:46:53.357875 19911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:53.364065 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 19840
I20260807 19:46:53.364257 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:46:53.364792 31552 external_mini_cluster.cc:1428] Setting key 8509a850adaa31e1f357a90c8d7e7cdf
I20260807 19:46:53.368912 19911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c: Bootstrap starting.
I20260807 19:46:53.371968 19911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:53.372941 19911 log.cc:824] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:53.375934 19911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c: No bootstrap required, opened a new log
I20260807 19:46:53.377888 19915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:53.366768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48686 (local address 127.30.208.62:37759)
0807 19:46:53.367723 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:53.367734 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:53.367775 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:46:53.369536 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:53.369558 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:53.369983 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:53.370359 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:53.370891 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:53.372159 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:53.373586 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:53.374269 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:53.374431 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:53.375550 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:53.375562 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:53.375580 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:53.376130 (+ 550us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:53.376217 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:53.376244 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:53.376359 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:53.376565 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 19:46:53.376855 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":329,"threads_started":1}
I20260807 19:46:53.383915 19911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d4af524ef4122a054be3646c40a5c" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37759 } }
I20260807 19:46:53.384270 19911 raft_consensus.cc:374] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:53.384380 19911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 573d4af524ef4122a054be3646c40a5c, State: Initialized, Role: FOLLOWER
I20260807 19:46:53.384950 19911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d4af524ef4122a054be3646c40a5c" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37759 } }
I20260807 19:46:53.385202 19911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:46:53.385366 19911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:46:53.385532 19911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:53.387844 19911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d4af524ef4122a054be3646c40a5c" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37759 } }
I20260807 19:46:53.388383 19911 leader_election.cc:302] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 573d4af524ef4122a054be3646c40a5c; no voters:
I20260807 19:46:53.389024 19911 leader_election.cc:288] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:46:53.389159 19917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:46:53.389889 19917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [term 1 LEADER]: Becoming Leader. State: Replica: 573d4af524ef4122a054be3646c40a5c, State: Running, Role: LEADER
I20260807 19:46:53.390434 19917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d4af524ef4122a054be3646c40a5c" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37759 } }
I20260807 19:46:53.391439 19911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:46:53.393374 19918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "573d4af524ef4122a054be3646c40a5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d4af524ef4122a054be3646c40a5c" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37759 } } }
I20260807 19:46:53.393872 19918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:53.393878 19919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 573d4af524ef4122a054be3646c40a5c. Latest consensus state: current_term: 1 leader_uuid: "573d4af524ef4122a054be3646c40a5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "573d4af524ef4122a054be3646c40a5c" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37759 } } }
I20260807 19:46:53.394193 19919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:53.395416 19925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:46:53.404742 19925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:46:53.413523 19925 catalog_manager.cc:1347] Generated new cluster ID: 12d127cbf2c04fd9b43d24ff1903b34e
I20260807 19:46:53.413651 19925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:46:53.426687 19925 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:46:53.427943 19925 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:46:53.441628 19925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c: Generated new TSK 0
I20260807 19:46:53.442528 19925 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:46:53.448273 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:37759
--builtin_ntp_servers=127.30.208.20:45159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:46:53.789855 19938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:53.790144 19938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:53.790259 19938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:53.797771 19938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:53.797871 19938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:53.798004 19938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:46:53.807636 19938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:37759
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:53.808947 19938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:53.810768 19938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:53.823397 19947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:53.823436 19945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:53.823457 19944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:53.824601 19938 server_base.cc:1034] running on GCE node
I20260807 19:46:53.825307 19938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:53.826314 19938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:53.827536 19938 hybrid_clock.cc:648] HybridClock initialized: now 1786132013827479 us; error 48 us; skew 500 ppm
I20260807 19:46:53.827855 19938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:46:53.830101 19938 webserver.cc:460] Webserver started at http://127.30.208.1:45817/ using document root <none> and password file <none>
I20260807 19:46:53.830760 19938 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:53.830865 19938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:53.831156 19938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:53.834450 19938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "12a318926b334637bfc869ac6b4e64f1"
format_stamp: "Formatted at 2026-08-07 19:46:53 on dist-test-slave-vq2q"
server_key: "0218149b7b851a2d92c072129969a126"
server_key_iv: "a4bc6c52d2a4dcdb70073f8e29370964"
server_key_version: "encryptionkey@0"
I20260807 19:46:53.835297 19938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "12a318926b334637bfc869ac6b4e64f1"
format_stamp: "Formatted at 2026-08-07 19:46:53 on dist-test-slave-vq2q"
server_key: "0218149b7b851a2d92c072129969a126"
server_key_iv: "a4bc6c52d2a4dcdb70073f8e29370964"
server_key_version: "encryptionkey@0"
I20260807 19:46:53.840642 19938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 19:46:53.844129 19953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:53.845510 19938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:46:53.845659 19938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "12a318926b334637bfc869ac6b4e64f1"
format_stamp: "Formatted at 2026-08-07 19:46:53 on dist-test-slave-vq2q"
server_key: "0218149b7b851a2d92c072129969a126"
server_key_iv: "a4bc6c52d2a4dcdb70073f8e29370964"
server_key_version: "encryptionkey@0"
I20260807 19:46:53.845875 19938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:53.862578 19938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:53.863392 19938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:53.863672 19938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:53.864606 19938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:53.866117 19938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:53.866218 19938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:53.866323 19938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:53.866401 19938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:53.894920 19938 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:45751
I20260807 19:46:53.895015 20077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:45751 every 8 connection(s)
I20260807 19:46:53.896289 19938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:46:53.897294 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 19938
I20260807 19:46:53.897460 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:46:53.897928 31552 external_mini_cluster.cc:1428] Setting key 28323eb151af3007b8ea5838b3438b0c
I20260807 19:46:53.901154 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:37759
--builtin_ntp_servers=127.30.208.20:45159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:46:53.908711 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:53.898852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:52847 (local address 127.30.208.62:37759)
0807 19:46:53.900886 (+ 2034us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:53.900892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:53.900913 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:46:53.902535 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:53.902541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:53.902619 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:53.902716 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:53.903766 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:53.904475 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:53.906041 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:53.906453 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:53.906576 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:53.907912 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:53.907917 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:53.907921 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:53.907939 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:53.907980 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:53.907992 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:53.908064 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:53.908273 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 19:46:53.908374 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1918,"thread_start_us":1817,"threads_started":1}
I20260807 19:46:53.909732 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:53.899407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37759 (local address 127.30.208.1:52847)
0807 19:46:53.900440 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:53.900492 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:53.902272 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:53.902962 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:53.902970 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:53.903055 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:53.903600 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:53.903616 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:53.904617 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:53.904620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:53.905898 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:53.905911 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:53.906669 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:53.906677 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:53.906913 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:53.907562 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:46:53.907565 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:53.907603 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:46:53.907676 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:53.908080 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:53.908092 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:53.908099 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 19:46:53.908248 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 19:46:53.908635 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":323,"threads_started":1}
I20260807 19:46:53.911230 20078 heartbeater.cc:347] Connected to a master server at 127.30.208.62:37759
I20260807 19:46:53.911561 20078 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:53.912320 20078 heartbeater.cc:511] Master 127.30.208.62:37759 requested a full tablet report, sending...
I20260807 19:46:53.914496 19873 ts_manager.cc:194] Registered new tserver with Master: 12a318926b334637bfc869ac6b4e64f1 (127.30.208.1:45751)
I20260807 19:46:53.916432 19873 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.208.1:52847
W20260807 19:46:54.240532 20082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:54.240818 20082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:54.240906 20082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:54.248216 20082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:54.248319 20082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:54.248425 20082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:46:54.257668 20082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:37759
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:54.258975 20082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:54.260730 20082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:54.273488 20090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:54.273468 20088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:54.273449 20087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:54.274307 20082 server_base.cc:1034] running on GCE node
I20260807 19:46:54.274964 20082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:54.276029 20082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:54.277287 20082 hybrid_clock.cc:648] HybridClock initialized: now 1786132014277243 us; error 35 us; skew 500 ppm
I20260807 19:46:54.277592 20082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:46:54.279914 20082 webserver.cc:460] Webserver started at http://127.30.208.2:34469/ using document root <none> and password file <none>
I20260807 19:46:54.280604 20082 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:54.280695 20082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:54.280963 20082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:54.283878 20082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "5106cc43d9564afd881cc22b01f1ff0e"
format_stamp: "Formatted at 2026-08-07 19:46:54 on dist-test-slave-vq2q"
server_key: "a9a8c08e30fc17af76f1b81338cb665e"
server_key_iv: "a08f751633c076966969a32f1b81f7af"
server_key_version: "encryptionkey@0"
I20260807 19:46:54.284652 20082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "5106cc43d9564afd881cc22b01f1ff0e"
format_stamp: "Formatted at 2026-08-07 19:46:54 on dist-test-slave-vq2q"
server_key: "a9a8c08e30fc17af76f1b81338cb665e"
server_key_iv: "a08f751633c076966969a32f1b81f7af"
server_key_version: "encryptionkey@0"
I20260807 19:46:54.289696 20082 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 19:46:54.293171 20096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:54.294471 20082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:46:54.294636 20082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "5106cc43d9564afd881cc22b01f1ff0e"
format_stamp: "Formatted at 2026-08-07 19:46:54 on dist-test-slave-vq2q"
server_key: "a9a8c08e30fc17af76f1b81338cb665e"
server_key_iv: "a08f751633c076966969a32f1b81f7af"
server_key_version: "encryptionkey@0"
I20260807 19:46:54.294834 20082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:54.310348 20082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:54.311179 20082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:54.311484 20082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:54.312451 20082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:54.313977 20082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:54.314076 20082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:54.314185 20082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:54.314265 20082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:54.346304 20082 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:34923
I20260807 19:46:54.346369 20220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:34923 every 8 connection(s)
I20260807 19:46:54.347621 20082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:46:54.352646 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 20082
I20260807 19:46:54.352856 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:46:54.353338 31552 external_mini_cluster.cc:1428] Setting key 8382eaa41ad63d855cdb923912e14c74
I20260807 19:46:54.356177 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:37759
--builtin_ntp_servers=127.30.208.20:45159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:46:54.362159 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:54.350242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:37443 (local address 127.30.208.62:37759)
0807 19:46:54.350457 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:54.350463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:54.351878 (+ 1415us) server_negotiation.cc:407] Connection header received
0807 19:46:54.354240 (+ 2362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:54.354246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:54.354346 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:54.354469 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:54.355769 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.356819 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.359140 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.359679 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.359815 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:54.361373 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:54.361379 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:54.361383 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:54.361405 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:54.361444 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:54.361459 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:54.361544 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:54.361756 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 19:46:54.361864 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:46:54.363602 20223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:54.350794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37759 (local address 127.30.208.2:37443)
0807 19:46:54.351725 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:54.351769 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:54.353949 (+ 2180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:54.354714 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:54.354727 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:54.354844 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:54.355567 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:54.355588 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.356984 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.356990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:54.358981 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:54.358997 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.359896 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.359903 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:54.360195 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:54.360991 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:46:54.360996 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:54.361048 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:46:54.361147 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:54.361549 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:54.361558 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:54.361565 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 19:46:54.361718 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 19:46:54.362109 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":304,"threads_started":1}
I20260807 19:46:54.365500 20221 heartbeater.cc:347] Connected to a master server at 127.30.208.62:37759
I20260807 19:46:54.365823 20221 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:54.366760 20221 heartbeater.cc:511] Master 127.30.208.62:37759 requested a full tablet report, sending...
I20260807 19:46:54.368239 19873 ts_manager.cc:194] Registered new tserver with Master: 5106cc43d9564afd881cc22b01f1ff0e (127.30.208.2:34923)
I20260807 19:46:54.369068 19873 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.208.2:37443
W20260807 19:46:54.706063 20224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:54.706408 20224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:54.706552 20224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:54.714200 20224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:54.714335 20224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:54.714497 20224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:46:54.724298 20224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:45159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:37759
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:54.725785 20224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:54.727772 20224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:54.741619 20229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:54.741549 20230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:54.741585 20232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:54.742620 20224 server_base.cc:1034] running on GCE node
I20260807 19:46:54.743305 20224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:54.744633 20224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:54.745973 20224 hybrid_clock.cc:648] HybridClock initialized: now 1786132014745899 us; error 52 us; skew 500 ppm
I20260807 19:46:54.746351 20224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:46:54.749008 20224 webserver.cc:460] Webserver started at http://127.30.208.3:35675/ using document root <none> and password file <none>
I20260807 19:46:54.749827 20224 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:54.749926 20224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:54.750242 20224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:54.753541 20224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "6429ddd3599a454bb905f1bbb0f3eace"
format_stamp: "Formatted at 2026-08-07 19:46:54 on dist-test-slave-vq2q"
server_key: "9f18f34577dd8bd4cbf8355cc17f4254"
server_key_iv: "ebb62cb5419bede66a4daea6ebe5c73b"
server_key_version: "encryptionkey@0"
I20260807 19:46:54.754369 20224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "6429ddd3599a454bb905f1bbb0f3eace"
format_stamp: "Formatted at 2026-08-07 19:46:54 on dist-test-slave-vq2q"
server_key: "9f18f34577dd8bd4cbf8355cc17f4254"
server_key_iv: "ebb62cb5419bede66a4daea6ebe5c73b"
server_key_version: "encryptionkey@0"
I20260807 19:46:54.759765 20224 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 19:46:54.763587 20239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:54.765271 20224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:46:54.765453 20224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "6429ddd3599a454bb905f1bbb0f3eace"
format_stamp: "Formatted at 2026-08-07 19:46:54 on dist-test-slave-vq2q"
server_key: "9f18f34577dd8bd4cbf8355cc17f4254"
server_key_iv: "ebb62cb5419bede66a4daea6ebe5c73b"
server_key_version: "encryptionkey@0"
I20260807 19:46:54.765677 20224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:54.795615 20224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:54.796484 20224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:54.796775 20224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:54.797827 20224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:54.799468 20224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:54.799568 20224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:54.799680 20224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:54.799758 20224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:54.830729 20224 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:34287
I20260807 19:46:54.830803 20363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:34287 every 8 connection(s)
I20260807 19:46:54.831981 20224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:46:54.839509 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 20224
I20260807 19:46:54.839670 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:46:54.840173 31552 external_mini_cluster.cc:1428] Setting key b532d96f5df7a1fee1d21f76eb55687e
I20260807 19:46:54.844985 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:54.834396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:34473 (local address 127.30.208.62:37759)
0807 19:46:54.834628 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:54.834634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:54.835987 (+ 1353us) server_negotiation.cc:407] Connection header received
0807 19:46:54.837802 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:54.837807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:54.837881 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:54.837970 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:54.839037 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.839861 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.841982 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.842554 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.842684 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:54.844137 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:54.844142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:54.844146 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:54.844165 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:54.844202 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:54.844215 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:54.844294 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:54.844634 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 19:46:54.844750 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 19:46:54.846031 20366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:54.834927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:37759 (local address 127.30.208.3:34473)
0807 19:46:54.835815 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:54.835857 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:54.837520 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:54.838168 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:54.838178 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:54.838287 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:54.838865 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:54.838883 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.840010 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.840015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:54.841827 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:54.841844 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.842823 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.842829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:54.843101 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:54.843809 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:46:54.843812 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:54.843854 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:46:54.843937 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:54.844409 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:54.844419 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:54.844426 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 19:46:54.844584 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 19:46:54.844926 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":289,"threads_started":1}
I20260807 19:46:54.847419 20364 heartbeater.cc:347] Connected to a master server at 127.30.208.62:37759
I20260807 19:46:54.847692 20364 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:54.848421 20364 heartbeater.cc:511] Master 127.30.208.62:37759 requested a full tablet report, sending...
I20260807 19:46:54.849931 19873 ts_manager.cc:194] Registered new tserver with Master: 6429ddd3599a454bb905f1bbb0f3eace (127.30.208.3:34287)
I20260807 19:46:54.850713 19873 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.208.3:34473
I20260807 19:46:54.855765 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:46:54.865971 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:54.859818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48702 (local address 127.30.208.62:37759)
0807 19:46:54.859986 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:54.859991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:54.860477 (+ 486us) server_negotiation.cc:407] Connection header received
0807 19:46:54.860647 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:54.860651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:54.860716 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:54.860842 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:54.861366 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.862071 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.863488 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.863878 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.863984 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:54.865125 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:54.865130 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:54.865134 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:54.865152 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:54.865211 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:54.865227 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:54.865386 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:54.865577 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 19:46:54.865640 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:46:54.870067 19873 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:46:54.876931 19873 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:46:54.912983 20377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:54.889275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:34287 (local address 127.0.0.1:36550)
0807 19:46:54.903974 (+ 14699us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:54.903993 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:54.904098 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:54.905542 (+ 1444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:54.905549 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:54.905578 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:54.906041 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:54.906056 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.907984 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.907989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:54.910127 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:54.910170 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.912039 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.912046 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:54.912266 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:54.912325 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 19:46:54.912447 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 19:46:54.912628 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14588,"mutex_wait_us":1,"thread_start_us":13251,"threads_started":1}
I20260807 19:46:54.913029 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:54.889647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36550 (local address 127.30.208.3:34287)
0807 19:46:54.904876 (+ 15229us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:54.904885 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:54.904923 (+ 38us) server_negotiation.cc:407] Connection header received
0807 19:46:54.905031 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:54.905039 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:54.905334 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:54.905525 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:54.906285 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.907851 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.910399 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.911835 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.911984 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:54.912271 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:54.912513 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 19:46:54.912677 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15042,"thread_start_us":14893,"threads_started":1}
I20260807 19:46:54.914361 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:54.889136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:45751 (local address 127.0.0.1:51910)
0807 19:46:54.902044 (+ 12908us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:54.902098 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:54.902323 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:54.907217 (+ 4894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:54.907222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:54.907250 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:54.907580 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:54.907591 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.909460 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.909466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:54.911814 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:54.911829 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.913588 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.913594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:54.913759 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:54.913789 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 19:46:54.913870 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 19:46:54.914003 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12566,"mutex_wait_us":118,"thread_start_us":14993,"threads_started":1}
I20260807 19:46:54.914549 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:54.889307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51910 (local address 127.30.208.1:45751)
0807 19:46:54.906204 (+ 16897us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:54.906212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:54.906237 (+ 25us) server_negotiation.cc:407] Connection header received
0807 19:46:54.906802 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:54.906812 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:54.907010 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:54.907765 (+ 755us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:54.908014 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.909299 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.912047 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.913402 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.913533 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:54.913787 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:54.914056 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 19:46:54.914211 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16735,"thread_start_us":16580,"threads_started":1}
I20260807 19:46:54.914661 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:54.904784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:34923 (local address 127.0.0.1:55460)
0807 19:46:54.906888 (+ 2104us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:54.906904 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:54.907092 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:54.907895 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:54.907898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:54.907919 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:54.908455 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:54.908473 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.910006 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.910010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:54.911998 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:54.912012 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.913931 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.913937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:54.914104 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:54.914134 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 19:46:54.914219 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 19:46:54.914357 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1986,"mutex_wait_us":57,"thread_start_us":3232,"threads_started":1}
I20260807 19:46:54.915315 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:54.904885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55460 (local address 127.30.208.2:34923)
0807 19:46:54.906517 (+ 1632us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:54.906526 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:54.906996 (+ 470us) server_negotiation.cc:407] Connection header received
0807 19:46:54.907282 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:54.907290 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:54.907514 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:54.907677 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:54.908657 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.909868 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.912224 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:54.913596 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:54.913716 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:54.913940 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:54.914835 (+ 895us) server_negotiation.cc:299] Negotiation successful
0807 19:46:54.914992 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1496,"thread_start_us":1366,"threads_started":1}
I20260807 19:46:54.917326 20292 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe7e3d5cc054b1aae0393f1de6e0d36 (DEFAULT_TABLE table=test-table [id=ee2d7811c4f64c07957ad0e98f582201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:54.918619 20292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe7e3d5cc054b1aae0393f1de6e0d36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:54.918826 20004 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe7e3d5cc054b1aae0393f1de6e0d36 (DEFAULT_TABLE table=test-table [id=ee2d7811c4f64c07957ad0e98f582201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:54.918826 20150 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe7e3d5cc054b1aae0393f1de6e0d36 (DEFAULT_TABLE table=test-table [id=ee2d7811c4f64c07957ad0e98f582201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:54.919657 20078 heartbeater.cc:503] Master 127.30.208.62:37759 was elected leader, sending a full tablet report...
I20260807 19:46:54.920197 20150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe7e3d5cc054b1aae0393f1de6e0d36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:54.920675 20004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe7e3d5cc054b1aae0393f1de6e0d36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:54.930899 20385 tablet_bootstrap.cc:492] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace: Bootstrap starting.
I20260807 19:46:54.934499 20386 tablet_bootstrap.cc:492] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1: Bootstrap starting.
I20260807 19:46:54.935671 20385 tablet_bootstrap.cc:654] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:54.936210 20387 tablet_bootstrap.cc:492] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e: Bootstrap starting.
I20260807 19:46:54.936941 20385 log.cc:824] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:54.938091 20386 tablet_bootstrap.cc:654] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:54.939109 20386 log.cc:824] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:54.939465 20387 tablet_bootstrap.cc:654] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:54.940359 20385 tablet_bootstrap.cc:492] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace: No bootstrap required, opened a new log
I20260807 19:46:54.940650 20385 ts_tablet_manager.cc:1397] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 19:46:54.940665 20387 log.cc:824] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:54.941805 20386 tablet_bootstrap.cc:492] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1: No bootstrap required, opened a new log
I20260807 19:46:54.942061 20386 ts_tablet_manager.cc:1397] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 19:46:54.943540 20387 tablet_bootstrap.cc:492] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e: No bootstrap required, opened a new log
I20260807 19:46:54.943801 20387 ts_tablet_manager.cc:1397] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 19:46:54.949275 20386 raft_consensus.cc:348] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a318926b334637bfc869ac6b4e64f1" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 45751 } } peers { permanent_uuid: "6429ddd3599a454bb905f1bbb0f3eace" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34287 } } peers { permanent_uuid: "5106cc43d9564afd881cc22b01f1ff0e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34923 } }
I20260807 19:46:54.949632 20386 raft_consensus.cc:374] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:54.949720 20386 raft_consensus.cc:729] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12a318926b334637bfc869ac6b4e64f1, State: Initialized, Role: FOLLOWER
I20260807 19:46:54.950352 20386 consensus_queue.cc:260] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a318926b334637bfc869ac6b4e64f1" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 45751 } } peers { permanent_uuid: "6429ddd3599a454bb905f1bbb0f3eace" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34287 } } peers { permanent_uuid: "5106cc43d9564afd881cc22b01f1ff0e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34923 } }
I20260807 19:46:54.951879 20386 ts_tablet_manager.cc:1428] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260807 19:46:54.952497 20385 raft_consensus.cc:348] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a318926b334637bfc869ac6b4e64f1" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 45751 } } peers { permanent_uuid: "6429ddd3599a454bb905f1bbb0f3eace" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34287 } } peers { permanent_uuid: "5106cc43d9564afd881cc22b01f1ff0e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34923 } }
I20260807 19:46:54.952898 20385 raft_consensus.cc:374] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:54.952993 20385 raft_consensus.cc:729] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6429ddd3599a454bb905f1bbb0f3eace, State: Initialized, Role: FOLLOWER
I20260807 19:46:54.953577 20385 consensus_queue.cc:260] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a318926b334637bfc869ac6b4e64f1" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 45751 } } peers { permanent_uuid: "6429ddd3599a454bb905f1bbb0f3eace" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34287 } } peers { permanent_uuid: "5106cc43d9564afd881cc22b01f1ff0e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34923 } }
I20260807 19:46:54.954486 20387 raft_consensus.cc:348] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a318926b334637bfc869ac6b4e64f1" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 45751 } } peers { permanent_uuid: "6429ddd3599a454bb905f1bbb0f3eace" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34287 } } peers { permanent_uuid: "5106cc43d9564afd881cc22b01f1ff0e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34923 } }
I20260807 19:46:54.954927 20387 raft_consensus.cc:374] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:54.955039 20387 raft_consensus.cc:729] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5106cc43d9564afd881cc22b01f1ff0e, State: Initialized, Role: FOLLOWER
I20260807 19:46:54.955111 20385 ts_tablet_manager.cc:1428] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260807 19:46:54.955710 20364 heartbeater.cc:503] Master 127.30.208.62:37759 was elected leader, sending a full tablet report...
I20260807 19:46:54.955560 20387 consensus_queue.cc:260] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a318926b334637bfc869ac6b4e64f1" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 45751 } } peers { permanent_uuid: "6429ddd3599a454bb905f1bbb0f3eace" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34287 } } peers { permanent_uuid: "5106cc43d9564afd881cc22b01f1ff0e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34923 } }
I20260807 19:46:54.956821 20221 heartbeater.cc:503] Master 127.30.208.62:37759 was elected leader, sending a full tablet report...
I20260807 19:46:54.957305 20387 ts_tablet_manager.cc:1428] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e: Time spent starting tablet: real 0.013s user 0.011s sys 0.004s
I20260807 19:46:55.021229 20392 raft_consensus.cc:482] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:55.021624 20392 raft_consensus.cc:504] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a318926b334637bfc869ac6b4e64f1" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 45751 } } peers { permanent_uuid: "6429ddd3599a454bb905f1bbb0f3eace" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34287 } } peers { permanent_uuid: "5106cc43d9564afd881cc22b01f1ff0e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34923 } }
I20260807 19:46:55.023389 20392 leader_election.cc:288] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12a318926b334637bfc869ac6b4e64f1 (127.30.208.1:45751), 5106cc43d9564afd881cc22b01f1ff0e (127.30.208.2:34923)
I20260807 19:46:55.029613 20366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.023743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:45751 (local address 127.30.208.3:43969)
0807 19:46:55.023975 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:55.023997 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:55.024152 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:55.024720 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:55.024725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:55.024755 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:55.024909 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:55.024917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.025912 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.025916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:55.027613 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:55.027628 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.028930 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.028936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:55.029095 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:55.029127 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:46:55.029273 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 19:46:55.029390 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":1}
I20260807 19:46:55.029690 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.023772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:43969 (local address 127.30.208.1:45751)
0807 19:46:55.023978 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:55.023984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:55.024089 (+ 105us) server_negotiation.cc:407] Connection header received
0807 19:46:55.024324 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:55.024330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:55.024419 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:55.024523 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:55.025053 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.025787 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.027796 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.028667 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.028835 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:55.028926 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:55.029311 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 19:46:55.029453 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:46:55.030580 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.024151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:34923 (local address 127.30.208.3:42599)
0807 19:46:55.024852 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:55.024869 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:55.024988 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:55.025335 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:55.025339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:55.025361 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:55.025706 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:55.025717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.026776 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.026782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:55.028377 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:55.028388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.029999 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.030004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:55.030172 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:55.030205 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:46:55.030294 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 19:46:55.030382 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":396,"threads_started":1}
I20260807 19:46:55.030644 20028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe7e3d5cc054b1aae0393f1de6e0d36" candidate_uuid: "6429ddd3599a454bb905f1bbb0f3eace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12a318926b334637bfc869ac6b4e64f1" is_pre_election: true
I20260807 19:46:55.030917 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.024483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:42599 (local address 127.30.208.2:34923)
0807 19:46:55.024684 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:55.024690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:55.024950 (+ 260us) server_negotiation.cc:407] Connection header received
0807 19:46:55.025119 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:55.025123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:55.025229 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:55.025311 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:55.025859 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.026618 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.028578 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.029681 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.029832 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:55.029897 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:55.030357 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 19:46:55.030497 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:46:55.031260 20028 raft_consensus.cc:2454] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6429ddd3599a454bb905f1bbb0f3eace in term 0.
I20260807 19:46:55.031788 20171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe7e3d5cc054b1aae0393f1de6e0d36" candidate_uuid: "6429ddd3599a454bb905f1bbb0f3eace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5106cc43d9564afd881cc22b01f1ff0e" is_pre_election: true
I20260807 19:46:55.031909 20240 leader_election.cc:302] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12a318926b334637bfc869ac6b4e64f1, 6429ddd3599a454bb905f1bbb0f3eace; no voters:
I20260807 19:46:55.032272 20392 raft_consensus.cc:2781] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:46:55.032305 20171 raft_consensus.cc:2454] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6429ddd3599a454bb905f1bbb0f3eace in term 0.
I20260807 19:46:55.032433 20392 raft_consensus.cc:482] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:46:55.032536 20392 raft_consensus.cc:3037] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:55.034811 20392 raft_consensus.cc:504] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a318926b334637bfc869ac6b4e64f1" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 45751 } } peers { permanent_uuid: "6429ddd3599a454bb905f1bbb0f3eace" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34287 } } peers { permanent_uuid: "5106cc43d9564afd881cc22b01f1ff0e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34923 } }
I20260807 19:46:55.035348 20392 leader_election.cc:288] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [CANDIDATE]: Term 1 election: Requested vote from peers 12a318926b334637bfc869ac6b4e64f1 (127.30.208.1:45751), 5106cc43d9564afd881cc22b01f1ff0e (127.30.208.2:34923)
I20260807 19:46:55.035837 20028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe7e3d5cc054b1aae0393f1de6e0d36" candidate_uuid: "6429ddd3599a454bb905f1bbb0f3eace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12a318926b334637bfc869ac6b4e64f1"
I20260807 19:46:55.035935 20171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe7e3d5cc054b1aae0393f1de6e0d36" candidate_uuid: "6429ddd3599a454bb905f1bbb0f3eace" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5106cc43d9564afd881cc22b01f1ff0e"
I20260807 19:46:55.036038 20028 raft_consensus.cc:3037] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:55.036119 20171 raft_consensus.cc:3037] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:55.038308 20028 raft_consensus.cc:2454] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6429ddd3599a454bb905f1bbb0f3eace in term 1.
I20260807 19:46:55.038391 20171 raft_consensus.cc:2454] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6429ddd3599a454bb905f1bbb0f3eace in term 1.
I20260807 19:46:55.038672 20240 leader_election.cc:302] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12a318926b334637bfc869ac6b4e64f1, 6429ddd3599a454bb905f1bbb0f3eace; no voters:
I20260807 19:46:55.038925 20392 raft_consensus.cc:2781] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:46:55.039336 20392 raft_consensus.cc:686] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [term 1 LEADER]: Becoming Leader. State: Replica: 6429ddd3599a454bb905f1bbb0f3eace, State: Running, Role: LEADER
I20260807 19:46:55.039911 20392 consensus_queue.cc:237] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a318926b334637bfc869ac6b4e64f1" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 45751 } } peers { permanent_uuid: "6429ddd3599a454bb905f1bbb0f3eace" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34287 } } peers { permanent_uuid: "5106cc43d9564afd881cc22b01f1ff0e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34923 } }
I20260807 19:46:55.044795 19873 catalog_manager.cc:5495] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace reported cstate change: term changed from 0 to 1, leader changed from <none> to 6429ddd3599a454bb905f1bbb0f3eace (127.30.208.3). New cstate: current_term: 1 leader_uuid: "6429ddd3599a454bb905f1bbb0f3eace" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12a318926b334637bfc869ac6b4e64f1" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 45751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6429ddd3599a454bb905f1bbb0f3eace" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 34287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5106cc43d9564afd881cc22b01f1ff0e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34923 } health_report { overall_health: UNKNOWN } } }
I20260807 19:46:55.070086 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.064536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36556 (local address 127.30.208.3:34287)
0807 19:46:55.064748 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:55.064763 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:55.064784 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:46:55.065024 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:55.065028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:55.065105 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:55.065265 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:46:55.065652 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.066398 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.067739 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.068155 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.068270 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:55.068853 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:46:55.069488 (+ 635us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:46:55.069644 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:55.069835 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 19:46:55.069878 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:46:55.078051 20171 raft_consensus.cc:1261] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e [term 1 FOLLOWER]: Refusing update from remote peer 6429ddd3599a454bb905f1bbb0f3eace: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:46:55.078051 20028 raft_consensus.cc:1261] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1 [term 1 FOLLOWER]: Refusing update from remote peer 6429ddd3599a454bb905f1bbb0f3eace: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:46:55.078969 20395 consensus_queue.cc:1035] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [LEADER]: Connected to new peer: Peer: permanent_uuid: "5106cc43d9564afd881cc22b01f1ff0e" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:46:55.079342 20392 consensus_queue.cc:1035] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [LEADER]: Connected to new peer: Peer: permanent_uuid: "12a318926b334637bfc869ac6b4e64f1" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 45751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:46:55.083492 20365 tablet.cc:2369] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:46:55.090288 20399 mvcc.cc:205] Tried to move back new op lower bound from 7315996733750108160 to 7315996733607432192. Current Snapshot: MvccSnapshot[applied={T|T < 7315996733750108160}]
I20260807 19:46:55.092475 20401 mvcc.cc:205] Tried to move back new op lower bound from 7315996733750108160 to 7315996733607432192. Current Snapshot: MvccSnapshot[applied={T|T < 7315996733750108160}]
I20260807 19:46:55.094349 20400 mvcc.cc:205] Tried to move back new op lower bound from 7315996733750108160 to 7315996733607432192. Current Snapshot: MvccSnapshot[applied={T|T < 7315996733750108160}]
I20260807 19:46:55.103335 19872 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:46:55.103590 19872 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:46:55.107326 19872 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 573d4af524ef4122a054be3646c40a5c: Sending DeleteTablet for 3 replicas of tablet 1fe7e3d5cc054b1aae0393f1de6e0d36
I20260807 19:46:55.108459 20292 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe7e3d5cc054b1aae0393f1de6e0d36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:55 UTC) from {username='slave'} at 127.0.0.1:36550
I20260807 19:46:55.108559 20004 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe7e3d5cc054b1aae0393f1de6e0d36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:55 UTC) from {username='slave'} at 127.0.0.1:51910
I20260807 19:46:55.108719 20150 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe7e3d5cc054b1aae0393f1de6e0d36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:55 UTC) from {username='slave'} at 127.0.0.1:55460
I20260807 19:46:55.110152 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 19938
I20260807 19:46:55.124114 20412 tablet_replica.cc:330] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e: stopping tablet replica
I20260807 19:46:55.124631 20412 raft_consensus.cc:2229] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:46:55.125201 20412 raft_consensus.cc:2258] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:55.127681 20412 ts_tablet_manager.cc:1905] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:55.135083 20412 ts_tablet_manager.cc:1918] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:55.135278 20412 log.cc:1196] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/1fe7e3d5cc054b1aae0393f1de6e0d36
I20260807 19:46:55.135870 20412 ts_tablet_manager.cc:1939] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 5106cc43d9564afd881cc22b01f1ff0e: Deleting consensus metadata
I20260807 19:46:55.137038 20411 tablet_replica.cc:330] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1: stopping tablet replica
I20260807 19:46:55.137655 20411 raft_consensus.cc:2229] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:46:55.138017 19857 catalog_manager.cc:4841] TS 5106cc43d9564afd881cc22b01f1ff0e (127.30.208.2:34923): tablet 1fe7e3d5cc054b1aae0393f1de6e0d36 (table test-table [id=ee2d7811c4f64c07957ad0e98f582201]) successfully deleted
I20260807 19:46:55.138473 20411 raft_consensus.cc:2258] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:55.139078 20409 tablet_replica.cc:330] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace: stopping tablet replica
I20260807 19:46:55.139519 20409 raft_consensus.cc:2229] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:46:55.140321 20409 raft_consensus.cc:2258] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:55.141628 20411 ts_tablet_manager.cc:1905] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 12a318926b334637bfc869ac6b4e64f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:55.142182 20409 ts_tablet_manager.cc:1905] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:55.144945 20379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.111228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51926 (local address 127.30.208.1:45751)
0807 19:46:55.111402 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:55.111407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:55.136444 (+ 25037us) server_negotiation.cc:407] Connection header received
0807 19:46:55.136542 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:55.136548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:55.136692 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:55.136828 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:55.137586 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.138833 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.140804 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.141474 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.141580 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:55.143135 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:55.143147 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:55.143168 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:55.143973 (+ 805us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:55.144035 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:55.144061 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:55.144159 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:55.144430 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 19:46:55.144561 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 19:46:55.146214 20072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:55.148816 20409 ts_tablet_manager.cc:1918] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:55.149010 20409 log.cc:1196] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/1fe7e3d5cc054b1aae0393f1de6e0d36
I20260807 19:46:55.149605 20409 ts_tablet_manager.cc:1939] T 1fe7e3d5cc054b1aae0393f1de6e0d36 P 6429ddd3599a454bb905f1bbb0f3eace: Deleting consensus metadata
I20260807 19:46:55.151569 19860 catalog_manager.cc:4841] TS 6429ddd3599a454bb905f1bbb0f3eace (127.30.208.3:34287): tablet 1fe7e3d5cc054b1aae0393f1de6e0d36 (table test-table [id=ee2d7811c4f64c07957ad0e98f582201]) successfully deleted
I20260807 19:46:55.238308 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 19938
I20260807 19:46:55.255659 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 20082
W20260807 19:46:55.255762 19857 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.208.1:45751 (error 108)
I20260807 19:46:55.257688 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.257238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:45751 (local address 127.0.0.1:51932)
0807 19:46:55.257381 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:55.257514 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:45751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 19:46:55.258934 19857 catalog_manager.cc:4584] TS 12a318926b334637bfc869ac6b4e64f1 (127.30.208.1:45751): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1fe7e3d5cc054b1aae0393f1de6e0d36: Network error: Client connection negotiation failed: client connection to 127.30.208.1:45751: connect: Connection refused (error 111)
I20260807 19:46:55.263062 20383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.256834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55468 (local address 127.30.208.2:34923)
0807 19:46:55.257012 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:55.257019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:55.257038 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:46:55.257104 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:55.257110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:55.257272 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:55.257384 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:55.257913 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.258663 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.260094 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.260506 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.260613 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:55.261719 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:55.261729 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:55.261742 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:55.262301 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:55.262349 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:55.262370 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:55.262474 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:55.262603 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 19:46:55.262741 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:46:55.263805 20216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:55.312510 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.312034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:45751 (local address 127.0.0.1:51936)
0807 19:46:55.312241 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:55.312345 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:45751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 19:46:55.313601 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.313265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:45751 (local address 127.0.0.1:51942)
0807 19:46:55.313407 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:55.313478 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:45751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 19:46:55.374375 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.373770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:45751 (local address 127.0.0.1:51950)
0807 19:46:55.374062 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:55.374170 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:45751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260807 19:46:55.375676 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.375251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:45751 (local address 127.0.0.1:51960)
0807 19:46:55.375451 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:55.375532 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:45751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 19:46:55.390286 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 20082
I20260807 19:46:55.407913 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 20224
I20260807 19:46:55.415073 20380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.408778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36560 (local address 127.30.208.3:34287)
0807 19:46:55.408952 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:55.408958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:55.408977 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:46:55.409146 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:55.409161 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:55.409298 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:55.409393 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:55.409850 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.410671 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.411948 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:55.412394 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:55.412519 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:55.413561 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:55.413573 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:55.413601 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:46:55.414274 (+ 673us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:55.414331 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:46:55.414355 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:55.414451 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:55.414595 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 19:46:55.414693 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:46:55.415771 20359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:55.456530 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.456023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:45751 (local address 127.0.0.1:51972)
0807 19:46:55.456281 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:55.456380 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:45751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 19:46:55.457556 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.457198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:45751 (local address 127.0.0.1:51988)
0807 19:46:55.457373 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:55.457443 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:45751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 19:46:55.538863 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 20224
I20260807 19:46:55.558676 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 19840
I20260807 19:46:55.559875 19906 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:55.671072 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:55.670534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:45751 (local address 127.0.0.1:52000)
0807 19:46:55.670749 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:55.670866 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:45751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 19:46:55.671132 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 19840
2026-08-07T19:46:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2832 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:46:55 dist-test-slave-vq2q krb5kdc[20421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:46:55 dist-test-slave-vq2q krb5kdc[20421](info): set up 2 sockets
Aug 07 19:46:55 dist-test-slave-vq2q krb5kdc[20421](info): commencing operation
krb5kdc: starting...
W20260807 19:46:56.772750 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:46:56 dist-test-slave-vq2q krb5kdc[20421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:46:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:46:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:46:57.072651 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:34675
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:34675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:57.410244 20436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:57.410550 20436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:57.410674 20436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:57.418026 20436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:46:57.418146 20436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:57.418219 20436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:57.418299 20436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:46:57.418371 20436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:46:57.428732 20436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:34675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:34675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:57.430112 20436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:57.431444 20436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:57.442788 20442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:57.442845 20446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:57.442806 20443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:57.443665 20436 server_base.cc:1034] running on GCE node
I20260807 19:46:57.444414 20436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:57.445919 20436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:57.447134 20436 hybrid_clock.cc:648] HybridClock initialized: now 1786132017447067 us; error 49 us; skew 500 ppm
I20260807 19:46:57.447463 20436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:57 dist-test-slave-vq2q krb5kdc[20421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132017, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:57.454797 20436 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:46:57.456344 20436 webserver.cc:460] Webserver started at http://127.30.208.62:35479/ using document root <none> and password file <none>
I20260807 19:46:57.457093 20436 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:57.457244 20436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:57.457566 20436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:57.460676 20436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "39a8a83cd3b14e8eaae864691bab433f"
format_stamp: "Formatted at 2026-08-07 19:46:57 on dist-test-slave-vq2q"
server_key: "e06145d9d0fd2d593b20c8c26cc685f3"
server_key_iv: "47f5deefc405530b7bc9d55e61947e51"
server_key_version: "encryptionkey@0"
I20260807 19:46:57.461493 20436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "39a8a83cd3b14e8eaae864691bab433f"
format_stamp: "Formatted at 2026-08-07 19:46:57 on dist-test-slave-vq2q"
server_key: "e06145d9d0fd2d593b20c8c26cc685f3"
server_key_iv: "47f5deefc405530b7bc9d55e61947e51"
server_key_version: "encryptionkey@0"
I20260807 19:46:57.466703 20436 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:46:57.470183 20453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:57.471524 20436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:46:57.471688 20436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "39a8a83cd3b14e8eaae864691bab433f"
format_stamp: "Formatted at 2026-08-07 19:46:57 on dist-test-slave-vq2q"
server_key: "e06145d9d0fd2d593b20c8c26cc685f3"
server_key_iv: "47f5deefc405530b7bc9d55e61947e51"
server_key_version: "encryptionkey@0"
I20260807 19:46:57.471870 20436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:57.497658 20436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:57.501828 20436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:57.502121 20436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:57.520016 20436 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:34675
I20260807 19:46:57.520035 20506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:34675 every 8 connection(s)
I20260807 19:46:57.521431 20436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:46:57.525139 20507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:57.531786 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 20436
I20260807 19:46:57.532011 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:46:57.532532 31552 external_mini_cluster.cc:1428] Setting key ca4b6ff3fad70773110ae2e846ecafd9
I20260807 19:46:57.536288 20507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f: Bootstrap starting.
I20260807 19:46:57.540056 20507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:57.541128 20507 log.cc:824] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:57.544358 20507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f: No bootstrap required, opened a new log
Aug 07 19:46:57 dist-test-slave-vq2q krb5kdc[20421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:57.553699 20507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39a8a83cd3b14e8eaae864691bab433f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34675 } }
I20260807 19:46:57.554082 20507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:57.554190 20507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39a8a83cd3b14e8eaae864691bab433f, State: Initialized, Role: FOLLOWER
I20260807 19:46:57.554826 20507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39a8a83cd3b14e8eaae864691bab433f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34675 } }
I20260807 19:46:57.555087 20507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:46:57.555259 20507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:46:57.555512 20507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:57.556612 20510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:57.534597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36582 (local address 127.30.208.62:34675)
0807 19:46:57.535360 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:57.535372 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:57.535413 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:46:57.537157 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:57.537214 (+ 57us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:57.537611 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:57.537970 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:57.539864 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:57.541253 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:57.542820 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:57.543488 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:57.543623 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:57.548628 (+ 5005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:57.548637 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:57.548654 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:57.548699 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:57.552027 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:57.552951 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:57.552959 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:57.552966 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:57.553151 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:57.553606 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:57.553611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:57.553613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:57.554246 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:57.554536 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:57.555059 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 19:46:57.555509 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":353,"threads_started":1}
I20260807 19:46:57.558010 20507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39a8a83cd3b14e8eaae864691bab433f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34675 } }
I20260807 19:46:57.558497 20507 leader_election.cc:302] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39a8a83cd3b14e8eaae864691bab433f; no voters:
I20260807 19:46:57.559082 20507 leader_election.cc:288] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:46:57.559229 20512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:46:57.559983 20512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [term 1 LEADER]: Becoming Leader. State: Replica: 39a8a83cd3b14e8eaae864691bab433f, State: Running, Role: LEADER
I20260807 19:46:57.560560 20512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39a8a83cd3b14e8eaae864691bab433f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34675 } }
I20260807 19:46:57.561137 20507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:46:57.563786 20514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39a8a83cd3b14e8eaae864691bab433f. Latest consensus state: current_term: 1 leader_uuid: "39a8a83cd3b14e8eaae864691bab433f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39a8a83cd3b14e8eaae864691bab433f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34675 } } }
I20260807 19:46:57.564157 20514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:57.564633 20513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39a8a83cd3b14e8eaae864691bab433f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39a8a83cd3b14e8eaae864691bab433f" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34675 } } }
I20260807 19:46:57.564975 20513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f [sys.catalog]: This master's current role is: LEADER
I20260807 19:46:57.565898 20519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:46:57.570533 20519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:46:57.579011 20519 catalog_manager.cc:1347] Generated new cluster ID: 7efa5c872d89430d9c3f916cb6d2724e
I20260807 19:46:57.579128 20519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:46:57.591229 20519 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:46:57.592995 20519 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:46:57.601945 20519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f: Generated new TSK 0
I20260807 19:46:57.602896 20519 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
Entry for principal kudu/127.30.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab.
I20260807 19:46:57.676116 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34675
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:58.031919 20534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:58.032229 20534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:58.032363 20534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:58.039888 20534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:58.040007 20534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:58.040154 20534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:46:58.050220 20534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.1.keytab
--principal=kudu/127.30.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34675
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:58.051549 20534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:58.052857 20534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:58.066335 20541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:58.066602 20539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:58.066910 20544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:58.067770 20534 server_base.cc:1034] running on GCE node
I20260807 19:46:58.068399 20534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:58.069455 20534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:58.070714 20534 hybrid_clock.cc:648] HybridClock initialized: now 1786132018070655 us; error 43 us; skew 500 ppm
I20260807 19:46:58.071161 20534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:58 dist-test-slave-vq2q krb5kdc[20421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:58.075743 20534 init.cc:377] Logged in from keytab as kudu/127.30.208.1@KRBTEST.COM (short username kudu)
I20260807 19:46:58.077332 20534 webserver.cc:460] Webserver started at http://127.30.208.1:41229/ using document root <none> and password file <none>
I20260807 19:46:58.078043 20534 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:58.078145 20534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:58.078445 20534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:58.082345 20534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "59c64587ec334cabba53e5541e993fc0"
format_stamp: "Formatted at 2026-08-07 19:46:58 on dist-test-slave-vq2q"
server_key: "bceae9bc537a262bf77096a5bb8f018a"
server_key_iv: "3b060b8be4d2af523f85fa5af93e0f37"
server_key_version: "encryptionkey@0"
I20260807 19:46:58.083523 20534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "59c64587ec334cabba53e5541e993fc0"
format_stamp: "Formatted at 2026-08-07 19:46:58 on dist-test-slave-vq2q"
server_key: "bceae9bc537a262bf77096a5bb8f018a"
server_key_iv: "3b060b8be4d2af523f85fa5af93e0f37"
server_key_version: "encryptionkey@0"
I20260807 19:46:58.090281 20534 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 19:46:58.094054 20551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:58.095629 20534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:46:58.095814 20534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "59c64587ec334cabba53e5541e993fc0"
format_stamp: "Formatted at 2026-08-07 19:46:58 on dist-test-slave-vq2q"
server_key: "bceae9bc537a262bf77096a5bb8f018a"
server_key_iv: "3b060b8be4d2af523f85fa5af93e0f37"
server_key_version: "encryptionkey@0"
I20260807 19:46:58.096032 20534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:58.114358 20534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:58.118739 20534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:58.119002 20534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:58.120000 20534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:58.121491 20534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:58.121570 20534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:58.121661 20534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:58.121730 20534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:58.154577 20534 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:36215
I20260807 19:46:58.154594 20673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:36215 every 8 connection(s)
I20260807 19:46:58.155907 20534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:46:58.159590 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 20534
I20260807 19:46:58.159828 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:46:58.160416 31552 external_mini_cluster.cc:1428] Setting key 96c0c39679500c01dd5abc8f91a52ba0
Aug 07 19:46:58 dist-test-slave-vq2q krb5kdc[20421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.1@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:58.177587 20676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:58.158488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:45315 (local address 127.30.208.62:34675)
0807 19:46:58.159097 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:58.159102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:58.160107 (+ 1005us) server_negotiation.cc:407] Connection header received
0807 19:46:58.162308 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:58.162314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:58.162396 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:58.162494 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:58.164377 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:58.165100 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:58.166798 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:58.167210 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:58.167345 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:58.171304 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:58.171310 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:58.171313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:58.171348 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:58.174050 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:58.174817 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:58.174822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:58.174826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:58.174990 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:58.175585 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:58.175589 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:58.175592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:58.176015 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:58.176162 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:58.176973 (+ 811us) server_negotiation.cc:299] Negotiation successful
0807 19:46:58.177251 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":363,"threads_started":1}
I20260807 19:46:58.178395 20677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:58.159043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34675 (local address 127.30.208.1:45315)
0807 19:46:58.159917 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:58.159972 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:58.162022 (+ 2050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:58.162745 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:58.162754 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:58.163474 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:58.164141 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:58.164161 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:58.165292 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:58.165296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:58.166634 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:58.166648 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:58.167463 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:58.167470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:58.167696 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:58.168380 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:58.168403 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:58.171012 (+ 2609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:58.174222 (+ 3210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:58.174232 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:58.174252 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:58.174695 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:58.175154 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:58.175160 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:58.175164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:58.175442 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:58.176138 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:58.176144 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:58.176556 (+ 412us) client_negotiation.cc:769] Sending connection context
0807 19:46:58.176804 (+ 248us) client_negotiation.cc:240] Negotiation successful
0807 19:46:58.177271 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":314,"threads_started":1}
I20260807 19:46:58.179956 20674 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34675
I20260807 19:46:58.180263 20674 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:58.181036 20674 heartbeater.cc:511] Master 127.30.208.62:34675 requested a full tablet report, sending...
I20260807 19:46:58.183233 20472 ts_manager.cc:194] Registered new tserver with Master: 59c64587ec334cabba53e5541e993fc0 (127.30.208.1:36215)
I20260807 19:46:58.184777 20472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.1@KRBTEST.COM'} at 127.30.208.1:45315
WARNING: no policy specified for kudu/127.30.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
Entry for principal kudu/127.30.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab.
I20260807 19:46:58.230523 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34675
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:58.578357 20680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:58.578646 20680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:58.578742 20680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:58.586203 20680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:58.586333 20680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:58.586483 20680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:46:58.596514 20680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.2.keytab
--principal=kudu/127.30.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34675
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:58.597908 20680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:58.599229 20680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:58.612619 20685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:58.612581 20688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:58.612671 20686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:58.619009 20680 server_base.cc:1034] running on GCE node
I20260807 19:46:58.619714 20680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:58.620803 20680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:58.622035 20680 hybrid_clock.cc:648] HybridClock initialized: now 1786132018621981 us; error 45 us; skew 500 ppm
I20260807 19:46:58.622365 20680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:58 dist-test-slave-vq2q krb5kdc[20421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:58.625655 20680 init.cc:377] Logged in from keytab as kudu/127.30.208.2@KRBTEST.COM (short username kudu)
I20260807 19:46:58.627223 20680 webserver.cc:460] Webserver started at http://127.30.208.2:44561/ using document root <none> and password file <none>
I20260807 19:46:58.627938 20680 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:58.628057 20680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:58.628365 20680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:58.631423 20680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "fc663977ccd342e491aca259069d9ffc"
format_stamp: "Formatted at 2026-08-07 19:46:58 on dist-test-slave-vq2q"
server_key: "fff0196dd01936b744058862b70fdcf4"
server_key_iv: "2510805a3b973578c20a806d879938be"
server_key_version: "encryptionkey@0"
I20260807 19:46:58.632187 20680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "fc663977ccd342e491aca259069d9ffc"
format_stamp: "Formatted at 2026-08-07 19:46:58 on dist-test-slave-vq2q"
server_key: "fff0196dd01936b744058862b70fdcf4"
server_key_iv: "2510805a3b973578c20a806d879938be"
server_key_version: "encryptionkey@0"
I20260807 19:46:58.637332 20680 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 19:46:58.640758 20695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:58.642145 20680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:46:58.642308 20680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "fc663977ccd342e491aca259069d9ffc"
format_stamp: "Formatted at 2026-08-07 19:46:58 on dist-test-slave-vq2q"
server_key: "fff0196dd01936b744058862b70fdcf4"
server_key_iv: "2510805a3b973578c20a806d879938be"
server_key_version: "encryptionkey@0"
I20260807 19:46:58.642501 20680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:58.662648 20680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:58.666708 20680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:58.666987 20680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:58.667949 20680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:58.669291 20680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:58.669368 20680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:58.669484 20680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:58.669538 20680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:58.699698 20680 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:34415
I20260807 19:46:58.699746 20815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:34415 every 8 connection(s)
I20260807 19:46:58.701046 20680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:46:58.701949 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 20680
I20260807 19:46:58.702150 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:46:58.702696 31552 external_mini_cluster.cc:1428] Setting key d5da3347fa331c9d6e2fa2489d25f6de
Aug 07 19:46:58 dist-test-slave-vq2q krb5kdc[20421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.2@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:58.722468 20818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:58.703911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:55603 (local address 127.30.208.62:34675)
0807 19:46:58.704413 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:58.704419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:58.705479 (+ 1060us) server_negotiation.cc:407] Connection header received
0807 19:46:58.707592 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:58.707598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:58.707687 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:58.707782 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:58.709817 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:58.710780 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:58.712380 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:58.712810 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:58.712938 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:58.716523 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:58.716528 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:58.716532 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:58.716569 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:58.719314 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:58.720091 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:58.720095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:58.720097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:58.720230 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:58.720663 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:58.720667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:58.720669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:58.720947 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:58.721085 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:58.721843 (+ 758us) server_negotiation.cc:299] Negotiation successful
0807 19:46:58.722078 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":260,"threads_started":1}
I20260807 19:46:58.723413 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:58.704530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34675 (local address 127.30.208.2:55603)
0807 19:46:58.705318 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:58.705374 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:58.707222 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:58.708171 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:58.708182 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:58.708790 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:58.709599 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:58.709626 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:58.710946 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:58.710950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:58.712224 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:58.712238 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:58.713058 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:58.713066 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:58.713333 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:58.713998 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:58.714027 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:58.716255 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:58.719472 (+ 3217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:58.719483 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:58.719501 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:58.719958 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:58.720339 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:58.720344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:58.720348 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:58.720544 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:58.721077 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:58.721085 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:58.721471 (+ 386us) client_negotiation.cc:769] Sending connection context
0807 19:46:58.721717 (+ 246us) client_negotiation.cc:240] Negotiation successful
0807 19:46:58.722284 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":4609,"threads_started":1}
I20260807 19:46:58.724805 20816 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34675
I20260807 19:46:58.725090 20816 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:58.725833 20816 heartbeater.cc:511] Master 127.30.208.62:34675 requested a full tablet report, sending...
I20260807 19:46:58.727337 20472 ts_manager.cc:194] Registered new tserver with Master: fc663977ccd342e491aca259069d9ffc (127.30.208.2:34415)
I20260807 19:46:58.728153 20472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.2@KRBTEST.COM'} at 127.30.208.2:55603
WARNING: no policy specified for kudu/127.30.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
Entry for principal kudu/127.30.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab.
I20260807 19:46:58.768350 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34675
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:39097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:46:59.104640 20823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:46:59.104920 20823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:46:59.105027 20823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:46:59.112453 20823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:46:59.112567 20823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:46:59.112689 20823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:46:59.122605 20823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:39097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.3.keytab
--principal=kudu/127.30.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:34675
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:46:59.123883 20823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:46:59.125139 20823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:46:59.138085 20830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:59.138177 20833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:46:59.138190 20831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:46:59.138880 20823 server_base.cc:1034] running on GCE node
I20260807 19:46:59.139472 20823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:46:59.140429 20823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:46:59.141626 20823 hybrid_clock.cc:648] HybridClock initialized: now 1786132019141583 us; error 37 us; skew 500 ppm
I20260807 19:46:59.141945 20823 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:46:59 dist-test-slave-vq2q krb5kdc[20421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132019, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:46:59.145076 20823 init.cc:377] Logged in from keytab as kudu/127.30.208.3@KRBTEST.COM (short username kudu)
I20260807 19:46:59.146639 20823 webserver.cc:460] Webserver started at http://127.30.208.3:46173/ using document root <none> and password file <none>
I20260807 19:46:59.147358 20823 fs_manager.cc:359] Metadata directory not provided
I20260807 19:46:59.147469 20823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:46:59.147804 20823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:46:59.150799 20823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "996ef05c5bec46449d3da3358f2786fd"
format_stamp: "Formatted at 2026-08-07 19:46:59 on dist-test-slave-vq2q"
server_key: "193230bbe630913d7591f64c2c2b2c40"
server_key_iv: "2bdeec2de3a55431a20d4f61e4007ad2"
server_key_version: "encryptionkey@0"
I20260807 19:46:59.151641 20823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "996ef05c5bec46449d3da3358f2786fd"
format_stamp: "Formatted at 2026-08-07 19:46:59 on dist-test-slave-vq2q"
server_key: "193230bbe630913d7591f64c2c2b2c40"
server_key_iv: "2bdeec2de3a55431a20d4f61e4007ad2"
server_key_version: "encryptionkey@0"
I20260807 19:46:59.157158 20823 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:46:59.160687 20840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:59.162171 20823 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 19:46:59.162364 20823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "996ef05c5bec46449d3da3358f2786fd"
format_stamp: "Formatted at 2026-08-07 19:46:59 on dist-test-slave-vq2q"
server_key: "193230bbe630913d7591f64c2c2b2c40"
server_key_iv: "2bdeec2de3a55431a20d4f61e4007ad2"
server_key_version: "encryptionkey@0"
I20260807 19:46:59.162567 20823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:46:59.178834 20823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:46:59.182910 20823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:46:59.183203 20823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:46:59.184183 20823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:46:59.185566 20823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:46:59.185640 20823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:59.185752 20823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:46:59.185799 20823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:46:59.188350 20674 heartbeater.cc:503] Master 127.30.208.62:34675 was elected leader, sending a full tablet report...
I20260807 19:46:59.216078 20823 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:38111
I20260807 19:46:59.216122 20961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:38111 every 8 connection(s)
I20260807 19:46:59.217417 20823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:46:59.219424 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 20823
I20260807 19:46:59.219630 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:46:59.220110 31552 external_mini_cluster.cc:1428] Setting key 33181a91cc1abb175fbbdc660601066a
Aug 07 19:46:59 dist-test-slave-vq2q krb5kdc[20421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132019, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.3@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:46:59.237783 20818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.220316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:56173 (local address 127.30.208.62:34675)
0807 19:46:59.220527 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:59.220533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:59.221772 (+ 1239us) server_negotiation.cc:407] Connection header received
0807 19:46:59.223764 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:59.223768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:59.223863 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:59.223951 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:59.225453 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.226194 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.227735 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.228145 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.228248 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.231841 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:59.231846 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:59.231860 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:59.231896 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:59.234903 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:59.235758 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:59.235762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:59.235764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:59.235899 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:59.236293 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:59.236296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:59.236297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:59.236572 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:59.236687 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:59.237381 (+ 694us) server_negotiation.cc:299] Negotiation successful
0807 19:46:59.237559 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:46:59.238807 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.220758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34675 (local address 127.30.208.3:56173)
0807 19:46:59.221559 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:59.221608 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:59.223502 (+ 1894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:59.224167 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:59.224176 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:59.224676 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:46:59.225298 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:59.225319 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.226362 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.226367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:59.227587 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:59.227601 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.228359 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.228364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:59.228567 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.229289 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:46:59.229312 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:46:59.231610 (+ 2298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:46:59.235048 (+ 3438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:59.235058 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:59.235075 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:59.235633 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:59.236004 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:46:59.236008 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:46:59.236010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:46:59.236203 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:46:59.236671 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:46:59.236677 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:46:59.237019 (+ 342us) client_negotiation.cc:769] Sending connection context
0807 19:46:59.237277 (+ 258us) client_negotiation.cc:240] Negotiation successful
0807 19:46:59.237687 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":281,"threads_started":1}
I20260807 19:46:59.240125 20963 heartbeater.cc:347] Connected to a master server at 127.30.208.62:34675
I20260807 19:46:59.240410 20963 heartbeater.cc:464] Registering TS with master...
I20260807 19:46:59.241092 20963 heartbeater.cc:511] Master 127.30.208.62:34675 requested a full tablet report, sending...
I20260807 19:46:59.242471 20472 ts_manager.cc:194] Registered new tserver with Master: 996ef05c5bec46449d3da3358f2786fd (127.30.208.3:38111)
I20260807 19:46:59.243253 20472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.208.3@KRBTEST.COM'} at 127.30.208.3:56173
I20260807 19:46:59.247720 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:46:59.264248 20818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.251884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36588 (local address 127.30.208.62:34675)
0807 19:46:59.252037 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:59.252041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:59.252427 (+ 386us) server_negotiation.cc:407] Connection header received
0807 19:46:59.252570 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:59.252574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:59.252633 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:59.252745 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:59.254168 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.254934 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.256254 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.256684 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.256784 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.259269 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:59.259276 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:59.259281 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:59.259317 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:59.261878 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:59.262513 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:59.262517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:59.262519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:59.262663 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:59.263004 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:59.263007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:59.263009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:59.263336 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:59.263551 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:59.263831 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 19:46:59.263999 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:46:59.268174 20472 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:46:59.275357 20472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:46:59.298031 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.287327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40936 (local address 127.30.208.2:34415)
0807 19:46:59.287899 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:59.287906 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:59.288812 (+ 906us) server_negotiation.cc:407] Connection header received
0807 19:46:59.288929 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:59.288938 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:59.289172 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:59.289409 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:59.290404 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.292138 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.295374 (+ 3236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.296491 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.296618 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.296820 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:59.297538 (+ 718us) server_negotiation.cc:299] Negotiation successful
0807 19:46:59.297707 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":291,"threads_started":1}
I20260807 19:46:59.298081 20978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.287093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:34415 (local address 127.0.0.1:40936)
0807 19:46:59.288337 (+ 1244us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:59.288399 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:59.288586 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:59.289672 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:59.289680 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:59.289712 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:59.290138 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:59.290151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.292339 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.292345 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:59.295143 (+ 2798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:59.295163 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.296978 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.296984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:59.297259 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.297346 (+ 87us) client_negotiation.cc:769] Sending connection context
0807 19:46:59.297481 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 19:46:59.297699 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1053,"mutex_wait_us":24,"thread_start_us":1576,"threads_started":1}
I20260807 19:46:59.299485 20976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.287681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:38111 (local address 127.0.0.1:45714)
0807 19:46:59.289055 (+ 1374us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:59.289072 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:59.289205 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:59.290166 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:59.290172 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:59.290203 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:59.290642 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:59.290653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.292777 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.292783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:59.296524 (+ 3741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:59.296537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.298588 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.298594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:59.298782 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.298816 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:46:59.298903 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 19:46:59.299033 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1246,"mutex_wait_us":1299,"thread_start_us":300,"threads_started":1}
I20260807 19:46:59.299892 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.288020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45714 (local address 127.30.208.3:38111)
0807 19:46:59.288792 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:59.288799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:59.289344 (+ 545us) server_negotiation.cc:407] Connection header received
0807 19:46:59.289458 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:59.289466 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:59.289676 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:59.289869 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:59.290857 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.292579 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.296715 (+ 4136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.298118 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.298252 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.298461 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:59.299343 (+ 882us) server_negotiation.cc:299] Negotiation successful
0807 19:46:59.299516 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":438,"threads_started":1}
I20260807 19:46:59.302539 20747 tablet_service.cc:1467] Processing CreateTablet for tablet adad95103ff1468a9a4f75a97855cea3 (DEFAULT_TABLE table=test-table [id=65d6f01bcdca4e8aa388fcc28004517f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:59.303460 20984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.289109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:36215 (local address 127.0.0.1:51998)
0807 19:46:59.296073 (+ 6964us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:59.296090 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:59.296214 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:59.297003 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:59.297007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:59.297033 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:59.297546 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:59.297558 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.299433 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.299439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:59.301390 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:59.301402 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.302666 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.302672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:59.302838 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.302872 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 19:46:59.302944 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 19:46:59.303074 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6843,"mutex_wait_us":50,"thread_start_us":6710,"threads_started":1}
I20260807 19:46:59.303490 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.289510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51998 (local address 127.30.208.1:36215)
0807 19:46:59.295650 (+ 6140us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:59.295660 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:59.296202 (+ 542us) server_negotiation.cc:407] Connection header received
0807 19:46:59.296400 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:59.296409 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:59.296659 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:59.296862 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:59.297739 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.299266 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.301546 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.302466 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.302591 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.302822 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:59.303021 (+ 199us) server_negotiation.cc:299] Negotiation successful
0807 19:46:59.303189 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5929,"thread_start_us":5743,"threads_started":1}
I20260807 19:46:59.303570 20893 tablet_service.cc:1467] Processing CreateTablet for tablet adad95103ff1468a9a4f75a97855cea3 (DEFAULT_TABLE table=test-table [id=65d6f01bcdca4e8aa388fcc28004517f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:59.304209 20747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adad95103ff1468a9a4f75a97855cea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:59.305038 20893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adad95103ff1468a9a4f75a97855cea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:59.307004 20603 tablet_service.cc:1467] Processing CreateTablet for tablet adad95103ff1468a9a4f75a97855cea3 (DEFAULT_TABLE table=test-table [id=65d6f01bcdca4e8aa388fcc28004517f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:46:59.308073 20603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adad95103ff1468a9a4f75a97855cea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:46:59.318789 20986 tablet_bootstrap.cc:492] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0: Bootstrap starting.
I20260807 19:46:59.321684 20987 tablet_bootstrap.cc:492] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd: Bootstrap starting.
I20260807 19:46:59.322935 20986 tablet_bootstrap.cc:654] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:59.324230 20986 log.cc:824] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:59.325002 20988 tablet_bootstrap.cc:492] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc: Bootstrap starting.
I20260807 19:46:59.325160 20987 tablet_bootstrap.cc:654] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:59.326391 20987 log.cc:824] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:59.327529 20986 tablet_bootstrap.cc:492] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0: No bootstrap required, opened a new log
I20260807 19:46:59.327868 20986 ts_tablet_manager.cc:1397] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260807 19:46:59.328933 20988 tablet_bootstrap.cc:654] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc: Neither blocks nor log segments found. Creating new log.
I20260807 19:46:59.329010 20987 tablet_bootstrap.cc:492] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd: No bootstrap required, opened a new log
I20260807 19:46:59.329267 20987 ts_tablet_manager.cc:1397] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260807 19:46:59.330149 20988 log.cc:824] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc: Log is configured to *not* fsync() on all Append() calls
I20260807 19:46:59.333138 20988 tablet_bootstrap.cc:492] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc: No bootstrap required, opened a new log
I20260807 19:46:59.333472 20988 ts_tablet_manager.cc:1397] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260807 19:46:59.337021 20987 raft_consensus.cc:348] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc663977ccd342e491aca259069d9ffc" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34415 } } peers { permanent_uuid: "59c64587ec334cabba53e5541e993fc0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36215 } } peers { permanent_uuid: "996ef05c5bec46449d3da3358f2786fd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38111 } }
I20260807 19:46:59.337445 20987 raft_consensus.cc:374] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:59.337549 20987 raft_consensus.cc:729] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 996ef05c5bec46449d3da3358f2786fd, State: Initialized, Role: FOLLOWER
I20260807 19:46:59.337898 20986 raft_consensus.cc:348] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc663977ccd342e491aca259069d9ffc" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34415 } } peers { permanent_uuid: "59c64587ec334cabba53e5541e993fc0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36215 } } peers { permanent_uuid: "996ef05c5bec46449d3da3358f2786fd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38111 } }
I20260807 19:46:59.338112 20987 consensus_queue.cc:260] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc663977ccd342e491aca259069d9ffc" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34415 } } peers { permanent_uuid: "59c64587ec334cabba53e5541e993fc0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36215 } } peers { permanent_uuid: "996ef05c5bec46449d3da3358f2786fd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38111 } }
I20260807 19:46:59.338299 20986 raft_consensus.cc:374] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:59.338433 20986 raft_consensus.cc:729] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59c64587ec334cabba53e5541e993fc0, State: Initialized, Role: FOLLOWER
I20260807 19:46:59.339041 20986 consensus_queue.cc:260] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc663977ccd342e491aca259069d9ffc" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34415 } } peers { permanent_uuid: "59c64587ec334cabba53e5541e993fc0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36215 } } peers { permanent_uuid: "996ef05c5bec46449d3da3358f2786fd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38111 } }
I20260807 19:46:59.340847 20986 ts_tablet_manager.cc:1428] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260807 19:46:59.343320 20988 raft_consensus.cc:348] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc663977ccd342e491aca259069d9ffc" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34415 } } peers { permanent_uuid: "59c64587ec334cabba53e5541e993fc0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36215 } } peers { permanent_uuid: "996ef05c5bec46449d3da3358f2786fd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38111 } }
I20260807 19:46:59.343811 20988 raft_consensus.cc:374] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:46:59.343935 20988 raft_consensus.cc:729] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc663977ccd342e491aca259069d9ffc, State: Initialized, Role: FOLLOWER
I20260807 19:46:59.344525 20988 consensus_queue.cc:260] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc663977ccd342e491aca259069d9ffc" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34415 } } peers { permanent_uuid: "59c64587ec334cabba53e5541e993fc0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36215 } } peers { permanent_uuid: "996ef05c5bec46449d3da3358f2786fd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38111 } }
I20260807 19:46:59.345602 20816 heartbeater.cc:503] Master 127.30.208.62:34675 was elected leader, sending a full tablet report...
I20260807 19:46:59.345639 20963 heartbeater.cc:503] Master 127.30.208.62:34675 was elected leader, sending a full tablet report...
I20260807 19:46:59.346408 20987 ts_tablet_manager.cc:1428] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd: Time spent starting tablet: real 0.017s user 0.015s sys 0.001s
I20260807 19:46:59.348489 20988 ts_tablet_manager.cc:1428] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260807 19:46:59.386783 20992 raft_consensus.cc:482] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:46:59.387095 20992 raft_consensus.cc:504] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc663977ccd342e491aca259069d9ffc" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34415 } } peers { permanent_uuid: "59c64587ec334cabba53e5541e993fc0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36215 } } peers { permanent_uuid: "996ef05c5bec46449d3da3358f2786fd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38111 } }
I20260807 19:46:59.388911 20992 leader_election.cc:288] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc663977ccd342e491aca259069d9ffc (127.30.208.2:34415), 59c64587ec334cabba53e5541e993fc0 (127.30.208.1:36215)
I20260807 19:46:59.394784 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.389420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:55289 (local address 127.30.208.2:34415)
0807 19:46:59.389656 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:59.389662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:59.389684 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:46:59.389885 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:59.389890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:59.389964 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:59.390049 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:59.390554 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.391248 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.392814 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.393948 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.394099 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.394175 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:59.394418 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 19:46:59.394541 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 19:46:59.394784 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.389288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:34415 (local address 127.30.208.3:55289)
0807 19:46:59.389529 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:59.389549 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:59.389679 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:59.390217 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:59.390224 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:59.390246 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:59.390411 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:59.390419 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.391390 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.391395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:59.392651 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:59.392664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.394091 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.394096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:59.394265 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.394290 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 19:46:59.394370 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 19:46:59.394504 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 19:46:59.395666 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.389874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:58037 (local address 127.30.208.1:36215)
0807 19:46:59.390091 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:59.390096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:59.390520 (+ 424us) server_negotiation.cc:407] Connection header received
0807 19:46:59.390676 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:59.390679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:59.390741 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:59.390813 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:46:59.391336 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.392004 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.393977 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.394750 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.394898 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.394969 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:59.395286 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 19:46:59.395436 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:46:59.395687 20995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.389681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:36215 (local address 127.30.208.3:58037)
0807 19:46:59.390419 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0807 19:46:59.390433 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:46:59.390554 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:46:59.390841 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:46:59.390845 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:46:59.390869 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:46:59.391187 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:59.391197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.392142 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.392146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:59.393815 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:46:59.393830 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.394947 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.394953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:46:59.395121 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.395154 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 19:46:59.395237 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 19:46:59.395361 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":409,"threads_started":1}
I20260807 19:46:59.396224 20769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adad95103ff1468a9a4f75a97855cea3" candidate_uuid: "996ef05c5bec46449d3da3358f2786fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc663977ccd342e491aca259069d9ffc" is_pre_election: true
I20260807 19:46:59.396845 20769 raft_consensus.cc:2454] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 996ef05c5bec46449d3da3358f2786fd in term 0.
I20260807 19:46:59.396965 20624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adad95103ff1468a9a4f75a97855cea3" candidate_uuid: "996ef05c5bec46449d3da3358f2786fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59c64587ec334cabba53e5541e993fc0" is_pre_election: true
I20260807 19:46:59.397547 20624 raft_consensus.cc:2454] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 996ef05c5bec46449d3da3358f2786fd in term 0.
I20260807 19:46:59.397624 20844 leader_election.cc:302] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 996ef05c5bec46449d3da3358f2786fd, fc663977ccd342e491aca259069d9ffc; no voters:
I20260807 19:46:59.397997 20992 raft_consensus.cc:2781] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:46:59.398131 20992 raft_consensus.cc:482] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:46:59.398242 20992 raft_consensus.cc:3037] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:59.400445 20992 raft_consensus.cc:504] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc663977ccd342e491aca259069d9ffc" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34415 } } peers { permanent_uuid: "59c64587ec334cabba53e5541e993fc0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36215 } } peers { permanent_uuid: "996ef05c5bec46449d3da3358f2786fd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38111 } }
I20260807 19:46:59.401010 20992 leader_election.cc:288] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [CANDIDATE]: Term 1 election: Requested vote from peers fc663977ccd342e491aca259069d9ffc (127.30.208.2:34415), 59c64587ec334cabba53e5541e993fc0 (127.30.208.1:36215)
I20260807 19:46:59.401549 20769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adad95103ff1468a9a4f75a97855cea3" candidate_uuid: "996ef05c5bec46449d3da3358f2786fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc663977ccd342e491aca259069d9ffc"
I20260807 19:46:59.401605 20624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adad95103ff1468a9a4f75a97855cea3" candidate_uuid: "996ef05c5bec46449d3da3358f2786fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59c64587ec334cabba53e5541e993fc0"
I20260807 19:46:59.401734 20769 raft_consensus.cc:3037] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:59.401805 20624 raft_consensus.cc:3037] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:46:59.403941 20769 raft_consensus.cc:2454] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 996ef05c5bec46449d3da3358f2786fd in term 1.
I20260807 19:46:59.404088 20624 raft_consensus.cc:2454] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 996ef05c5bec46449d3da3358f2786fd in term 1.
I20260807 19:46:59.404342 20844 leader_election.cc:302] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 996ef05c5bec46449d3da3358f2786fd, fc663977ccd342e491aca259069d9ffc; no voters:
I20260807 19:46:59.404589 20992 raft_consensus.cc:2781] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:46:59.405057 20992 raft_consensus.cc:686] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [term 1 LEADER]: Becoming Leader. State: Replica: 996ef05c5bec46449d3da3358f2786fd, State: Running, Role: LEADER
I20260807 19:46:59.405618 20992 consensus_queue.cc:237] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc663977ccd342e491aca259069d9ffc" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34415 } } peers { permanent_uuid: "59c64587ec334cabba53e5541e993fc0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36215 } } peers { permanent_uuid: "996ef05c5bec46449d3da3358f2786fd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38111 } }
W20260807 19:46:59.408412 20675 tablet.cc:2369] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:46:59.410456 20472 catalog_manager.cc:5495] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 996ef05c5bec46449d3da3358f2786fd (127.30.208.3). New cstate: current_term: 1 leader_uuid: "996ef05c5bec46449d3da3358f2786fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc663977ccd342e491aca259069d9ffc" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59c64587ec334cabba53e5541e993fc0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "996ef05c5bec46449d3da3358f2786fd" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 38111 } health_report { overall_health: HEALTHY } } }
I20260807 19:46:59.433138 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.427877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45720 (local address 127.30.208.3:38111)
0807 19:46:59.428100 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:59.428106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:59.428125 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:46:59.428371 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:59.428375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:59.428461 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:59.428589 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:46:59.428974 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.429741 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.431121 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.431506 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.431616 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.432075 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:46:59.432581 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:46:59.432713 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:59.432813 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 19:46:59.432893 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:46:59.440881 20769 raft_consensus.cc:1261] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc [term 1 FOLLOWER]: Refusing update from remote peer 996ef05c5bec46449d3da3358f2786fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:46:59.440881 20624 raft_consensus.cc:1261] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0 [term 1 FOLLOWER]: Refusing update from remote peer 996ef05c5bec46449d3da3358f2786fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:46:59.441802 20996 consensus_queue.cc:1035] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "59c64587ec334cabba53e5541e993fc0" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 36215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:46:59.442126 20992 consensus_queue.cc:1035] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc663977ccd342e491aca259069d9ffc" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:46:59.453169 20817 tablet.cc:2369] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:46:59.467103 20472 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:46:59.467373 20472 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:46:59.470851 20472 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39a8a83cd3b14e8eaae864691bab433f: Sending DeleteTablet for 3 replicas of tablet adad95103ff1468a9a4f75a97855cea3
I20260807 19:46:59.471992 20893 tablet_service.cc:1514] Processing DeleteTablet for tablet adad95103ff1468a9a4f75a97855cea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:59 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:45714
I20260807 19:46:59.472090 20603 tablet_service.cc:1514] Processing DeleteTablet for tablet adad95103ff1468a9a4f75a97855cea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:59 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:51998
I20260807 19:46:59.472728 20747 tablet_service.cc:1514] Processing DeleteTablet for tablet adad95103ff1468a9a4f75a97855cea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:46:59 UTC) from {username='kudu', principal='kudu/127.30.208.62@KRBTEST.COM'} at 127.0.0.1:40936
I20260807 19:46:59.472931 21011 tablet_replica.cc:330] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd: stopping tablet replica
I20260807 19:46:59.473364 21011 raft_consensus.cc:2229] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:46:59.474068 21011 raft_consensus.cc:2258] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:59.474596 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 20534
I20260807 19:46:59.474622 21012 tablet_replica.cc:330] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0: stopping tablet replica
I20260807 19:46:59.475224 21012 raft_consensus.cc:2229] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:46:59.475739 21012 raft_consensus.cc:2258] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:59.477903 21012 ts_tablet_manager.cc:1905] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:59.478961 21011 ts_tablet_manager.cc:1905] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:59.484862 21012 ts_tablet_manager.cc:1918] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:59.485102 21012 log.cc:1196] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-0/wal/wals/adad95103ff1468a9a4f75a97855cea3
I20260807 19:46:59.485755 21012 ts_tablet_manager.cc:1939] T adad95103ff1468a9a4f75a97855cea3 P 59c64587ec334cabba53e5541e993fc0: Deleting consensus metadata
I20260807 19:46:59.487704 20457 catalog_manager.cc:4841] TS 59c64587ec334cabba53e5541e993fc0 (127.30.208.1:36215): tablet adad95103ff1468a9a4f75a97855cea3 (table test-table [id=65d6f01bcdca4e8aa388fcc28004517f]) successfully deleted
I20260807 19:46:59.488128 21014 tablet_replica.cc:330] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc: stopping tablet replica
I20260807 19:46:59.488490 21014 raft_consensus.cc:2229] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:46:59.488902 21014 raft_consensus.cc:2258] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:46:59.492969 21011 ts_tablet_manager.cc:1918] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:59.493155 21011 log.cc:1196] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/adad95103ff1468a9a4f75a97855cea3
I20260807 19:46:59.493777 21011 ts_tablet_manager.cc:1939] T adad95103ff1468a9a4f75a97855cea3 P 996ef05c5bec46449d3da3358f2786fd: Deleting consensus metadata
I20260807 19:46:59.495653 20455 catalog_manager.cc:4841] TS 996ef05c5bec46449d3da3358f2786fd (127.30.208.3:38111): tablet adad95103ff1468a9a4f75a97855cea3 (table test-table [id=65d6f01bcdca4e8aa388fcc28004517f]) successfully deleted
Aug 07 19:46:59 dist-test-slave-vq2q krb5kdc[20421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.1@KRBTEST.COM
I20260807 19:46:59.496981 21014 ts_tablet_manager.cc:1905] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:46:59.502478 21014 ts_tablet_manager.cc:1918] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:46:59.502718 21014 log.cc:1196] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/adad95103ff1468a9a4f75a97855cea3
I20260807 19:46:59.503232 21014 ts_tablet_manager.cc:1939] T adad95103ff1468a9a4f75a97855cea3 P fc663977ccd342e491aca259069d9ffc: Deleting consensus metadata
I20260807 19:46:59.505004 20457 catalog_manager.cc:4841] TS fc663977ccd342e491aca259069d9ffc (127.30.208.2:34415): tablet adad95103ff1468a9a4f75a97855cea3 (table test-table [id=65d6f01bcdca4e8aa388fcc28004517f]) successfully deleted
I20260807 19:46:59.505791 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.475719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52004 (local address 127.30.208.1:36215)
0807 19:46:59.475891 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:59.475897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:59.477689 (+ 1792us) server_negotiation.cc:407] Connection header received
0807 19:46:59.485547 (+ 7858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:59.485554 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:59.485649 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:59.485743 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:59.489326 (+ 3583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.490108 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.491382 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.492015 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.492147 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.497658 (+ 5511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:59.497676 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:59.497704 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:59.497749 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:59.501706 (+ 3957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:59.502730 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:59.502741 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:59.502745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:59.502922 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:59.503427 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:59.503433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:59.503436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:59.503896 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:59.504227 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:59.504549 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 19:46:59.505381 (+ 832us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 19:46:59.506422 20666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:59.595520 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 20534
I20260807 19:46:59.613536 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 20680
Aug 07 19:46:59 dist-test-slave-vq2q krb5kdc[20421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.2@KRBTEST.COM
I20260807 19:46:59.631770 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.614315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40940 (local address 127.30.208.2:34415)
0807 19:46:59.614477 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:59.614482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:59.614499 (+ 17us) server_negotiation.cc:407] Connection header received
0807 19:46:59.614632 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:59.614636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:59.614709 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:59.614791 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:59.616364 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.617064 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.618921 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.619320 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.619412 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.624583 (+ 5171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:59.624603 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:59.624624 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:59.624672 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:59.628634 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:59.629458 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:59.629468 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:59.629472 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:59.629643 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:59.630095 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:59.630100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:59.630103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:59.630592 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:59.630900 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:59.631136 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 19:46:59.631363 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:46:59.632495 20811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:59.721290 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 20680
I20260807 19:46:59.739457 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 20823
Aug 07 19:46:59 dist-test-slave-vq2q krb5kdc[20421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.3@KRBTEST.COM
I20260807 19:46:59.757923 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:46:59.740269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57198 (local address 127.30.208.3:38111)
0807 19:46:59.740503 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 19:46:59.740510 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:46:59.740533 (+ 23us) server_negotiation.cc:407] Connection header received
0807 19:46:59.740666 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:46:59.740672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:46:59.740774 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:46:59.740897 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:46:59.742423 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.743661 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.745067 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:46:59.745829 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:46:59.745972 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:46:59.750424 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:46:59.750440 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:46:59.750463 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:46:59.750540 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:46:59.754610 (+ 4070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:59.755403 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:59.755417 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:59.755421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:59.755619 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:46:59.756103 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:46:59.756109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:46:59.756112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:46:59.756610 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:46:59.756966 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0807 19:46:59.757212 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 19:46:59.757476 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 19:46:59.758852 20957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:46:59.882653 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 20823
I20260807 19:46:59.902815 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 20436
I20260807 19:46:59.903890 20502 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:00.015103 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 20436
2026-08-07T19:47:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4344 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T19:47:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:00Z Disabled control of system clock
I20260807 19:47:00.101656 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:44435
--webserver_interface=127.30.208.62
--webserver_port=0
--builtin_ntp_servers=127.30.208.20:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:44435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:47:00.439003 21023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:00.439285 21023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:00.439376 21023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:00.446584 21023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:00.446686 21023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:00.446743 21023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:00.446833 21023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:00.446885 21023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:00.455957 21023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:44435
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:44435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:00.457291 21023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:00.459031 21023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:00.470436 21029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:00.470520 21030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:00.470399 21032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:00.471120 21023 server_base.cc:1034] running on GCE node
I20260807 19:47:00.471930 21023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:00.473389 21023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:00.474587 21023 hybrid_clock.cc:648] HybridClock initialized: now 1786132020474545 us; error 37 us; skew 500 ppm
I20260807 19:47:00.474956 21023 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:47:00.477160 21023 webserver.cc:460] Webserver started at http://127.30.208.62:39583/ using document root <none> and password file <none>
I20260807 19:47:00.477890 21023 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:00.477995 21023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:00.478292 21023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:00.481387 21023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "326392304e804ad689e15da87d655d5a"
format_stamp: "Formatted at 2026-08-07 19:47:00 on dist-test-slave-vq2q"
server_key: "2044844570e01494dccd4dfc912905c0"
server_key_iv: "efc19f75aa0964f70cf20e9df83f192a"
server_key_version: "encryptionkey@0"
I20260807 19:47:00.482244 21023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "326392304e804ad689e15da87d655d5a"
format_stamp: "Formatted at 2026-08-07 19:47:00 on dist-test-slave-vq2q"
server_key: "2044844570e01494dccd4dfc912905c0"
server_key_iv: "efc19f75aa0964f70cf20e9df83f192a"
server_key_version: "encryptionkey@0"
I20260807 19:47:00.487401 21023 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260807 19:47:00.490900 21038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:00.492409 21023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:47:00.492571 21023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "326392304e804ad689e15da87d655d5a"
format_stamp: "Formatted at 2026-08-07 19:47:00 on dist-test-slave-vq2q"
server_key: "2044844570e01494dccd4dfc912905c0"
server_key_iv: "efc19f75aa0964f70cf20e9df83f192a"
server_key_version: "encryptionkey@0"
I20260807 19:47:00.492762 21023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:00.515661 21023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:00.516512 21023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:00.516769 21023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:00.535171 21023 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:44435
I20260807 19:47:00.535190 21095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:44435 every 8 connection(s)
I20260807 19:47:00.536525 21023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:00.538615 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 21023
I20260807 19:47:00.538774 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:00.539255 31552 external_mini_cluster.cc:1428] Setting key 0a6eae6f5aca3ebef6e767d6bb032fea
I20260807 19:47:00.540656 21096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:00.554605 21099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:00.541095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36728 (local address 127.30.208.62:44435)
0807 19:47:00.542207 (+ 1112us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:00.542222 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:00.542285 (+ 63us) server_negotiation.cc:407] Connection header received
0807 19:47:00.544120 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:00.544148 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:00.544758 (+ 610us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:00.545249 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:00.545808 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:00.547841 (+ 2033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:00.549244 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:00.550235 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:00.550413 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:00.551563 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:00.551574 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:00.551596 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:00.552378 (+ 782us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:00.552476 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:00.552502 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:00.552629 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:00.552872 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 19:47:00.553276 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":348,"threads_started":1}
I20260807 19:47:00.555655 21096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a: Bootstrap starting.
I20260807 19:47:00.558756 21096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:00.559741 21096 log.cc:824] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:00.562681 21096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a: No bootstrap required, opened a new log
I20260807 19:47:00.569917 21096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326392304e804ad689e15da87d655d5a" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44435 } }
I20260807 19:47:00.570320 21096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:00.570420 21096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326392304e804ad689e15da87d655d5a, State: Initialized, Role: FOLLOWER
I20260807 19:47:00.570957 21096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326392304e804ad689e15da87d655d5a" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44435 } }
I20260807 19:47:00.571175 21096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:00.571309 21096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:00.571460 21096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:00.573652 21096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326392304e804ad689e15da87d655d5a" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44435 } }
I20260807 19:47:00.574162 21096 leader_election.cc:302] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 326392304e804ad689e15da87d655d5a; no voters:
I20260807 19:47:00.574751 21096 leader_election.cc:288] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:00.574909 21101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:00.575999 21101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [term 1 LEADER]: Becoming Leader. State: Replica: 326392304e804ad689e15da87d655d5a, State: Running, Role: LEADER
I20260807 19:47:00.576591 21101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326392304e804ad689e15da87d655d5a" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44435 } }
I20260807 19:47:00.576833 21096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:00.579344 21102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "326392304e804ad689e15da87d655d5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326392304e804ad689e15da87d655d5a" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44435 } } }
I20260807 19:47:00.579304 21103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 326392304e804ad689e15da87d655d5a. Latest consensus state: current_term: 1 leader_uuid: "326392304e804ad689e15da87d655d5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "326392304e804ad689e15da87d655d5a" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 44435 } } }
I20260807 19:47:00.579775 21102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:00.579876 21103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:00.584146 21109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:00.589716 21109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:00.598743 21109 catalog_manager.cc:1347] Generated new cluster ID: 9dbe2b2cfbc84fc7a786a50b57d18b09
I20260807 19:47:00.598873 21109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:00.623978 21109 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:00.625898 21109 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:00.637336 21109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a: Generated new TSK 0
I20260807 19:47:00.638445 21109 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:00.647101 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.1:0
--local_ip_for_outbound_sockets=127.30.208.1
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44435
--builtin_ntp_servers=127.30.208.20:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 19:47:00.988886 21122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:00.989168 21122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:00.989328 21122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:00.996876 21122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:00.996999 21122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:00.997210 21122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.1
I20260807 19:47:01.006661 21122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.208.1
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.1
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:01.007954 21122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:01.009789 21122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:01.022918 21128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:01.022848 21131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:01.022879 21129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:01.023790 21122 server_base.cc:1034] running on GCE node
I20260807 19:47:01.024375 21122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:01.025386 21122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:01.026589 21122 hybrid_clock.cc:648] HybridClock initialized: now 1786132021026541 us; error 41 us; skew 500 ppm
I20260807 19:47:01.026899 21122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:47:01.029208 21122 webserver.cc:460] Webserver started at http://127.30.208.1:37765/ using document root <none> and password file <none>
I20260807 19:47:01.029893 21122 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:01.030000 21122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:01.030301 21122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:01.033510 21122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/data/instance:
uuid: "22ccd5f3595f420f9c941864a5c0f5db"
format_stamp: "Formatted at 2026-08-07 19:47:01 on dist-test-slave-vq2q"
server_key: "36eabc7116cb3474046064f15d222165"
server_key_iv: "8746b9538fe87e9d9052ff2438d69bae"
server_key_version: "encryptionkey@0"
I20260807 19:47:01.034394 21122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance:
uuid: "22ccd5f3595f420f9c941864a5c0f5db"
format_stamp: "Formatted at 2026-08-07 19:47:01 on dist-test-slave-vq2q"
server_key: "36eabc7116cb3474046064f15d222165"
server_key_iv: "8746b9538fe87e9d9052ff2438d69bae"
server_key_version: "encryptionkey@0"
I20260807 19:47:01.039417 21122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 19:47:01.043043 21137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:01.044358 21122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 19:47:01.044507 21122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/wal
uuid: "22ccd5f3595f420f9c941864a5c0f5db"
format_stamp: "Formatted at 2026-08-07 19:47:01 on dist-test-slave-vq2q"
server_key: "36eabc7116cb3474046064f15d222165"
server_key_iv: "8746b9538fe87e9d9052ff2438d69bae"
server_key_version: "encryptionkey@0"
I20260807 19:47:01.044713 21122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:01.075569 21122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:01.076453 21122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:01.076782 21122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:01.077894 21122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:01.079406 21122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:01.079502 21122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:01.079613 21122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:01.079681 21122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:01.111071 21122 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.1:39287
I20260807 19:47:01.111145 21260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.1:39287 every 8 connection(s)
I20260807 19:47:01.112318 21122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/data/info.pb
I20260807 19:47:01.120250 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 21122
I20260807 19:47:01.120428 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-0/wal/instance
I20260807 19:47:01.120857 31552 external_mini_cluster.cc:1428] Setting key 1cc0965b3ce11e5e2e4a4edb77080b4f
I20260807 19:47:01.123788 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.2:0
--local_ip_for_outbound_sockets=127.30.208.2
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44435
--builtin_ntp_servers=127.30.208.20:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:47:01.124715 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:01.114716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.1:34141 (local address 127.30.208.62:44435)
0807 19:47:01.115168 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:01.115172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:01.116212 (+ 1040us) server_negotiation.cc:407] Connection header received
0807 19:47:01.118063 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:01.118068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:01.118153 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:01.118253 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:01.119383 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:01.120107 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:01.121915 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:01.122344 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:01.122460 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:01.123828 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:01.123833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:01.123837 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:01.123862 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:01.123915 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:01.123930 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:01.124010 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:01.124266 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 19:47:01.124385 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":243,"threads_started":1}
I20260807 19:47:01.125808 21265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:01.115190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:44435 (local address 127.30.208.1:34141)
0807 19:47:01.116057 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:01.116094 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:01.117786 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:01.118544 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:01.118554 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:01.118645 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:01.119208 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:01.119225 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:01.120260 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:01.120264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:01.121762 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:01.121776 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:01.122573 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:01.122580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:01.122819 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:01.123516 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:47:01.123519 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:01.123561 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:47:01.123639 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:01.124061 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:01.124071 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:01.124077 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 19:47:01.124232 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 19:47:01.124633 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":267,"threads_started":1}
I20260807 19:47:01.127470 21261 heartbeater.cc:347] Connected to a master server at 127.30.208.62:44435
I20260807 19:47:01.127756 21261 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:01.128513 21261 heartbeater.cc:511] Master 127.30.208.62:44435 requested a full tablet report, sending...
I20260807 19:47:01.130930 21057 ts_manager.cc:194] Registered new tserver with Master: 22ccd5f3595f420f9c941864a5c0f5db (127.30.208.1:39287)
I20260807 19:47:01.132853 21057 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.208.1:34141
W20260807 19:47:01.460975 21266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:01.461274 21266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:01.461373 21266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:01.468560 21266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:01.468660 21266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:01.468768 21266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.2
I20260807 19:47:01.477865 21266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.208.2
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.2
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:01.479091 21266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:01.480769 21266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:01.493462 21277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:01.493510 21274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:01.493547 21275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:01.494700 21266 server_base.cc:1034] running on GCE node
I20260807 19:47:01.495313 21266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:01.496284 21266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:01.497511 21266 hybrid_clock.cc:648] HybridClock initialized: now 1786132021497464 us; error 41 us; skew 500 ppm
I20260807 19:47:01.497816 21266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:47:01.500085 21266 webserver.cc:460] Webserver started at http://127.30.208.2:34163/ using document root <none> and password file <none>
I20260807 19:47:01.500753 21266 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:01.500852 21266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:01.501149 21266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:01.504026 21266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/data/instance:
uuid: "b2268017eaed418180fbdb46965e83db"
format_stamp: "Formatted at 2026-08-07 19:47:01 on dist-test-slave-vq2q"
server_key: "5026b21a1b8a2c7fc604a61152f7cdf0"
server_key_iv: "cc024a0e5b2403ddd1aba14be96e5ec5"
server_key_version: "encryptionkey@0"
I20260807 19:47:01.504863 21266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance:
uuid: "b2268017eaed418180fbdb46965e83db"
format_stamp: "Formatted at 2026-08-07 19:47:01 on dist-test-slave-vq2q"
server_key: "5026b21a1b8a2c7fc604a61152f7cdf0"
server_key_iv: "cc024a0e5b2403ddd1aba14be96e5ec5"
server_key_version: "encryptionkey@0"
I20260807 19:47:01.510058 21266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260807 19:47:01.513407 21283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:01.514676 21266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 19:47:01.514829 21266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/wal
uuid: "b2268017eaed418180fbdb46965e83db"
format_stamp: "Formatted at 2026-08-07 19:47:01 on dist-test-slave-vq2q"
server_key: "5026b21a1b8a2c7fc604a61152f7cdf0"
server_key_iv: "cc024a0e5b2403ddd1aba14be96e5ec5"
server_key_version: "encryptionkey@0"
I20260807 19:47:01.515095 21266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:01.545167 21266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:01.546051 21266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:01.546331 21266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:01.547300 21266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:01.548796 21266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:01.548882 21266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:01.548990 21266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:01.549057 21266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:01.578689 21266 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.2:34465
I20260807 19:47:01.578763 21408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.2:34465 every 8 connection(s)
I20260807 19:47:01.579933 21266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/data/info.pb
I20260807 19:47:01.586673 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 21266
I20260807 19:47:01.586845 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/wal/instance
I20260807 19:47:01.587267 31552 external_mini_cluster.cc:1428] Setting key 7a0c983031a00655ec2e8c3b78dde7da
I20260807 19:47:01.590169 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.208.3:0
--local_ip_for_outbound_sockets=127.30.208.3
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44435
--builtin_ntp_servers=127.30.208.20:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 19:47:01.592388 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:01.582410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.2:58937 (local address 127.30.208.62:44435)
0807 19:47:01.582585 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:01.582590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:01.583979 (+ 1389us) server_negotiation.cc:407] Connection header received
0807 19:47:01.585830 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:01.585836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:01.585918 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:01.586004 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:01.587080 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:01.587880 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:01.589792 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:01.590180 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:01.590300 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:01.591704 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:01.591709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:01.591712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:01.591729 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:01.591765 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:01.591779 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:01.591850 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:01.592071 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 19:47:01.592150 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:47:01.593704 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:01.582927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:44435 (local address 127.30.208.2:58937)
0807 19:47:01.583825 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:01.583864 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:01.585560 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:01.586240 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:01.586248 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:01.586332 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:01.586918 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:01.586942 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:01.588072 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:01.588077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:01.589640 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:01.589655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:01.590409 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:01.590416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:01.590672 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:01.591367 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:47:01.591371 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:01.591424 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:47:01.591520 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:01.591873 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:01.591884 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:01.591892 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 19:47:01.592065 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 19:47:01.592431 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":262,"threads_started":1}
I20260807 19:47:01.595063 21409 heartbeater.cc:347] Connected to a master server at 127.30.208.62:44435
I20260807 19:47:01.595363 21409 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:01.596099 21409 heartbeater.cc:511] Master 127.30.208.62:44435 requested a full tablet report, sending...
I20260807 19:47:01.597500 21057 ts_manager.cc:194] Registered new tserver with Master: b2268017eaed418180fbdb46965e83db (127.30.208.2:34465)
I20260807 19:47:01.598269 21057 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.208.2:58937
W20260807 19:47:01.929239 21412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:01.929549 21412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:01.929688 21412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:01.937086 21412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:01.937249 21412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:01.937402 21412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.208.3
I20260807 19:47:01.946853 21412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.208.3
--webserver_port=0
--tserver_master_addrs=127.30.208.62:44435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.208.3
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:01.948267 21412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:01.950397 21412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:01.963160 21417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:01.963160 21418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:01.963230 21420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:01.964042 21412 server_base.cc:1034] running on GCE node
I20260807 19:47:01.964601 21412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:01.965544 21412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:01.966727 21412 hybrid_clock.cc:648] HybridClock initialized: now 1786132021966668 us; error 60 us; skew 500 ppm
I20260807 19:47:01.967010 21412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 19:47:01.969200 21412 webserver.cc:460] Webserver started at http://127.30.208.3:40841/ using document root <none> and password file <none>
I20260807 19:47:01.969866 21412 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:01.969954 21412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:01.970220 21412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:01.973066 21412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/data/instance:
uuid: "b743f853697c4862a645b0f2daf59c88"
format_stamp: "Formatted at 2026-08-07 19:47:01 on dist-test-slave-vq2q"
server_key: "f462289354a63beff273db5a902f247f"
server_key_iv: "74e3ce8773173ca1ce4883aa87d23e9c"
server_key_version: "encryptionkey@0"
I20260807 19:47:01.973889 21412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance:
uuid: "b743f853697c4862a645b0f2daf59c88"
format_stamp: "Formatted at 2026-08-07 19:47:01 on dist-test-slave-vq2q"
server_key: "f462289354a63beff273db5a902f247f"
server_key_iv: "74e3ce8773173ca1ce4883aa87d23e9c"
server_key_version: "encryptionkey@0"
I20260807 19:47:01.979070 21412 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:47:01.982533 21426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:01.983935 21412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:47:01.984099 21412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/wal
uuid: "b743f853697c4862a645b0f2daf59c88"
format_stamp: "Formatted at 2026-08-07 19:47:01 on dist-test-slave-vq2q"
server_key: "f462289354a63beff273db5a902f247f"
server_key_iv: "74e3ce8773173ca1ce4883aa87d23e9c"
server_key_version: "encryptionkey@0"
I20260807 19:47:01.984319 21412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:02.003379 21412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:02.004271 21412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:02.004593 21412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:02.005642 21412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 19:47:02.007292 21412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 19:47:02.007403 21412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:02.007508 21412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 19:47:02.007586 21412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:02.037886 21412 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.3:45779
I20260807 19:47:02.037955 21561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.3:45779 every 8 connection(s)
I20260807 19:47:02.039155 21412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/data/info.pb
I20260807 19:47:02.040380 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 21412
I20260807 19:47:02.040556 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/wal/instance
I20260807 19:47:02.041074 31552 external_mini_cluster.cc:1428] Setting key de4802b97e8c11c5d859f170ba050e55
I20260807 19:47:02.051913 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.041796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:41779 (local address 127.30.208.62:44435)
0807 19:47:02.042027 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:02.042032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:02.043720 (+ 1688us) server_negotiation.cc:407] Connection header received
0807 19:47:02.045714 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:02.045719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:02.045788 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:02.045874 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:02.046899 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.047726 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.049424 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.049898 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.049995 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.051235 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:02.051239 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:02.051243 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:02.051262 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:02.051296 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:02.051308 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:02.051377 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:02.051547 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 19:47:02.051632 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 19:47:02.052883 21564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.042414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:44435 (local address 127.30.208.3:41779)
0807 19:47:02.043551 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:02.043600 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:02.045440 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:02.046076 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:02.046084 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:02.046171 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:02.046733 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:02.046749 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.047906 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.047910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:02.049274 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:02.049288 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.050092 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.050097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:02.050297 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.050935 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 19:47:02.050938 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:02.050977 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 19:47:02.051060 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:02.051389 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:02.051396 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:02.051401 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 19:47:02.051533 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 19:47:02.051818 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":372,"threads_started":1}
I20260807 19:47:02.054272 21562 heartbeater.cc:347] Connected to a master server at 127.30.208.62:44435
I20260807 19:47:02.054580 21562 heartbeater.cc:464] Registering TS with master...
I20260807 19:47:02.055656 21562 heartbeater.cc:511] Master 127.30.208.62:44435 requested a full tablet report, sending...
I20260807 19:47:02.057322 21057 ts_manager.cc:194] Registered new tserver with Master: b743f853697c4862a645b0f2daf59c88 (127.30.208.3:45779)
I20260807 19:47:02.058121 21057 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.208.3:41779
I20260807 19:47:02.069520 31552 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 19:47:02.079270 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.073330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36730 (local address 127.30.208.62:44435)
0807 19:47:02.073531 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:02.073538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:02.073971 (+ 433us) server_negotiation.cc:407] Connection header received
0807 19:47:02.074142 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:02.074148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:02.074239 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:02.074325 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:02.074837 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.075553 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.076951 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.077424 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.077543 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.078548 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:02.078553 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:02.078556 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:02.078573 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:02.078606 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:02.078617 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:02.078690 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:02.078878 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 19:47:02.078973 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 19:47:02.083222 21057 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 19:47:02.091388 21057 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 19:47:02.115378 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.104054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39287 (local address 127.0.0.1:44762)
0807 19:47:02.105975 (+ 1921us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:02.106016 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:02.106533 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:02.107582 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:02.107589 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:02.107617 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:02.108034 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:02.108050 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.109919 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.109924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:02.112211 (+ 2287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:02.112227 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.114429 (+ 2202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.114435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:02.114644 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.114696 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 19:47:02.114845 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 19:47:02.115038 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1640,"thread_start_us":1458,"threads_started":1}
I20260807 19:47:02.116637 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.104267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.30.208.1:39287)
0807 19:47:02.105374 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:02.105384 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:02.106190 (+ 806us) server_negotiation.cc:407] Connection header received
0807 19:47:02.106805 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:02.106813 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:02.107058 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:02.107238 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:02.108222 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.109754 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.112410 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.113869 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.113979 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.114243 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:02.116062 (+ 1819us) server_negotiation.cc:299] Negotiation successful
0807 19:47:02.116243 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":959,"thread_start_us":827,"threads_started":1}
I20260807 19:47:02.124657 21579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.104441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.3:45779 (local address 127.0.0.1:53200)
0807 19:47:02.106603 (+ 2162us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:02.106618 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:02.106799 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:02.108765 (+ 1966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:02.108769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:02.108794 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:02.109136 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:02.109148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.117980 (+ 8832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.117985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:02.119825 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:02.119845 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.123833 (+ 3988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.123840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:02.124045 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.124081 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 19:47:02.124177 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 19:47:02.124315 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1963,"thread_start_us":1592,"threads_started":1}
I20260807 19:47:02.124050 21192 tablet_service.cc:1467] Processing CreateTablet for tablet d555c2ecc0974c01a66aab13c7fe01ca (DEFAULT_TABLE table=test-table [id=61edba72a98b438ca37bbb5f825c13ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:02.125100 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.104727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53200 (local address 127.30.208.3:45779)
0807 19:47:02.105811 (+ 1084us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:02.105819 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:02.106696 (+ 877us) server_negotiation.cc:407] Connection header received
0807 19:47:02.108083 (+ 1387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:02.108091 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:02.108311 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:02.108468 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:02.109385 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.117788 (+ 8403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.120040 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.123609 (+ 3569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.123753 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.124030 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:02.124240 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 19:47:02.124429 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":901,"thread_start_us":708,"threads_started":1}
I20260807 19:47:02.125747 21192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d555c2ecc0974c01a66aab13c7fe01ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:02.126811 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.106364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:34465 (local address 127.0.0.1:58756)
0807 19:47:02.115824 (+ 9460us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:02.115844 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:02.115945 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:02.121053 (+ 5108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:02.121057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:02.121087 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:02.121306 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:02.121316 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.123046 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.123050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:02.124481 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:02.124496 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.126059 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.126065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:02.126250 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.126282 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:47:02.126386 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 19:47:02.126524 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9361,"thread_start_us":13559,"threads_started":1}
I20260807 19:47:02.127689 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.106690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58756 (local address 127.30.208.2:34465)
0807 19:47:02.120433 (+ 13743us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:02.120442 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:02.120475 (+ 33us) server_negotiation.cc:407] Connection header received
0807 19:47:02.120584 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:02.120592 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:02.120825 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:02.121041 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:02.121450 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.122908 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.124674 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.125898 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.126032 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.126257 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:02.127210 (+ 953us) server_negotiation.cc:299] Negotiation successful
0807 19:47:02.127381 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13554,"thread_start_us":13437,"threads_started":1}
I20260807 19:47:02.128994 21486 tablet_service.cc:1467] Processing CreateTablet for tablet d555c2ecc0974c01a66aab13c7fe01ca (DEFAULT_TABLE table=test-table [id=61edba72a98b438ca37bbb5f825c13ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:02.130790 21486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d555c2ecc0974c01a66aab13c7fe01ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:02.130676 21337 tablet_service.cc:1467] Processing CreateTablet for tablet d555c2ecc0974c01a66aab13c7fe01ca (DEFAULT_TABLE table=test-table [id=61edba72a98b438ca37bbb5f825c13ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 19:47:02.131770 21337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d555c2ecc0974c01a66aab13c7fe01ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:02.138238 21261 heartbeater.cc:503] Master 127.30.208.62:44435 was elected leader, sending a full tablet report...
I20260807 19:47:02.142308 21585 tablet_bootstrap.cc:492] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db: Bootstrap starting.
I20260807 19:47:02.145321 21587 tablet_bootstrap.cc:492] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88: Bootstrap starting.
I20260807 19:47:02.146557 21585 tablet_bootstrap.cc:654] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:02.146775 21586 tablet_bootstrap.cc:492] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db: Bootstrap starting.
I20260807 19:47:02.148259 21585 log.cc:824] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:02.148852 21587 tablet_bootstrap.cc:654] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:02.150082 21587 log.cc:824] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:02.150247 21586 tablet_bootstrap.cc:654] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:02.151502 21586 log.cc:824] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:02.151916 21585 tablet_bootstrap.cc:492] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db: No bootstrap required, opened a new log
I20260807 19:47:02.152209 21585 ts_tablet_manager.cc:1397] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 19:47:02.154384 21587 tablet_bootstrap.cc:492] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88: No bootstrap required, opened a new log
I20260807 19:47:02.154660 21587 ts_tablet_manager.cc:1397] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260807 19:47:02.154850 21586 tablet_bootstrap.cc:492] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db: No bootstrap required, opened a new log
I20260807 19:47:02.155120 21586 ts_tablet_manager.cc:1397] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260807 19:47:02.159930 21585 raft_consensus.cc:348] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ccd5f3595f420f9c941864a5c0f5db" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39287 } } peers { permanent_uuid: "b743f853697c4862a645b0f2daf59c88" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 45779 } } peers { permanent_uuid: "b2268017eaed418180fbdb46965e83db" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34465 } }
I20260807 19:47:02.160398 21585 raft_consensus.cc:374] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:02.160497 21585 raft_consensus.cc:729] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ccd5f3595f420f9c941864a5c0f5db, State: Initialized, Role: FOLLOWER
I20260807 19:47:02.161250 21585 consensus_queue.cc:260] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ccd5f3595f420f9c941864a5c0f5db" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39287 } } peers { permanent_uuid: "b743f853697c4862a645b0f2daf59c88" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 45779 } } peers { permanent_uuid: "b2268017eaed418180fbdb46965e83db" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34465 } }
I20260807 19:47:02.163018 21585 ts_tablet_manager.cc:1428] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260807 19:47:02.165225 21586 raft_consensus.cc:348] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ccd5f3595f420f9c941864a5c0f5db" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39287 } } peers { permanent_uuid: "b743f853697c4862a645b0f2daf59c88" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 45779 } } peers { permanent_uuid: "b2268017eaed418180fbdb46965e83db" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34465 } }
I20260807 19:47:02.165230 21587 raft_consensus.cc:348] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ccd5f3595f420f9c941864a5c0f5db" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39287 } } peers { permanent_uuid: "b743f853697c4862a645b0f2daf59c88" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 45779 } } peers { permanent_uuid: "b2268017eaed418180fbdb46965e83db" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34465 } }
I20260807 19:47:02.165690 21587 raft_consensus.cc:374] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:02.165690 21586 raft_consensus.cc:374] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:02.165795 21587 raft_consensus.cc:729] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b743f853697c4862a645b0f2daf59c88, State: Initialized, Role: FOLLOWER
I20260807 19:47:02.165796 21586 raft_consensus.cc:729] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2268017eaed418180fbdb46965e83db, State: Initialized, Role: FOLLOWER
I20260807 19:47:02.166419 21587 consensus_queue.cc:260] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ccd5f3595f420f9c941864a5c0f5db" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39287 } } peers { permanent_uuid: "b743f853697c4862a645b0f2daf59c88" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 45779 } } peers { permanent_uuid: "b2268017eaed418180fbdb46965e83db" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34465 } }
I20260807 19:47:02.166419 21586 consensus_queue.cc:260] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ccd5f3595f420f9c941864a5c0f5db" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39287 } } peers { permanent_uuid: "b743f853697c4862a645b0f2daf59c88" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 45779 } } peers { permanent_uuid: "b2268017eaed418180fbdb46965e83db" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34465 } }
I20260807 19:47:02.167584 21409 heartbeater.cc:503] Master 127.30.208.62:44435 was elected leader, sending a full tablet report...
I20260807 19:47:02.168313 21586 ts_tablet_manager.cc:1428] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260807 19:47:02.169538 21562 heartbeater.cc:503] Master 127.30.208.62:44435 was elected leader, sending a full tablet report...
I20260807 19:47:02.170235 21587 ts_tablet_manager.cc:1428] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260807 19:47:02.263262 21593 raft_consensus.cc:482] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 19:47:02.263667 21593 raft_consensus.cc:504] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ccd5f3595f420f9c941864a5c0f5db" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39287 } } peers { permanent_uuid: "b743f853697c4862a645b0f2daf59c88" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 45779 } } peers { permanent_uuid: "b2268017eaed418180fbdb46965e83db" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34465 } }
I20260807 19:47:02.265655 21593 leader_election.cc:288] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ccd5f3595f420f9c941864a5c0f5db (127.30.208.1:39287), b2268017eaed418180fbdb46965e83db (127.30.208.2:34465)
I20260807 19:47:02.272019 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.266170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:49193 (local address 127.30.208.1:39287)
0807 19:47:02.266369 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:02.266375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:02.266394 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:47:02.266539 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:02.266545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:02.266618 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:02.266694 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:02.267184 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.267919 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.270332 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.271203 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.271331 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.271391 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:02.271668 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 19:47:02.271768 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 19:47:02.272107 21564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.266038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39287 (local address 127.30.208.3:49193)
0807 19:47:02.266217 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:02.266243 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:02.266396 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:02.266753 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:02.266758 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:02.266786 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:02.266996 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:02.267009 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.268058 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.268063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:02.270168 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:02.270184 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.271343 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.271349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:02.271517 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.271552 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 19:47:02.271647 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 19:47:02.271792 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 19:47:02.272992 21594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.266497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.2:34465 (local address 127.30.208.3:45939)
0807 19:47:02.267295 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:02.267309 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:02.267414 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:02.267706 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:02.267710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:02.267733 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 19:47:02.268064 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:02.268074 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.269048 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.269054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:02.270845 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:02.270856 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.272351 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.272358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:02.272532 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.272564 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 19:47:02.272653 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 19:47:02.272786 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":479,"threads_started":1}
I20260807 19:47:02.273098 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.266649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.208.3:45939 (local address 127.30.208.2:34465)
0807 19:47:02.266832 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:02.266839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:02.267397 (+ 558us) server_negotiation.cc:407] Connection header received
0807 19:47:02.267525 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:02.267529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:02.267607 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:02.267680 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 19:47:02.268254 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.268923 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.271029 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.272081 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.272227 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.272306 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:02.272712 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 19:47:02.272845 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:47:02.273312 21212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d555c2ecc0974c01a66aab13c7fe01ca" candidate_uuid: "b743f853697c4862a645b0f2daf59c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ccd5f3595f420f9c941864a5c0f5db" is_pre_election: true
I20260807 19:47:02.273892 21212 raft_consensus.cc:2454] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b743f853697c4862a645b0f2daf59c88 in term 0.
I20260807 19:47:02.274101 21361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d555c2ecc0974c01a66aab13c7fe01ca" candidate_uuid: "b743f853697c4862a645b0f2daf59c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2268017eaed418180fbdb46965e83db" is_pre_election: true
I20260807 19:47:02.274587 21430 leader_election.cc:302] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ccd5f3595f420f9c941864a5c0f5db, b743f853697c4862a645b0f2daf59c88; no voters:
I20260807 19:47:02.274592 21361 raft_consensus.cc:2454] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b743f853697c4862a645b0f2daf59c88 in term 0.
I20260807 19:47:02.275000 21593 raft_consensus.cc:2781] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 19:47:02.275135 21593 raft_consensus.cc:482] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 19:47:02.275235 21593 raft_consensus.cc:3037] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:02.277561 21593 raft_consensus.cc:504] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ccd5f3595f420f9c941864a5c0f5db" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39287 } } peers { permanent_uuid: "b743f853697c4862a645b0f2daf59c88" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 45779 } } peers { permanent_uuid: "b2268017eaed418180fbdb46965e83db" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34465 } }
I20260807 19:47:02.278118 21593 leader_election.cc:288] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [CANDIDATE]: Term 1 election: Requested vote from peers 22ccd5f3595f420f9c941864a5c0f5db (127.30.208.1:39287), b2268017eaed418180fbdb46965e83db (127.30.208.2:34465)
I20260807 19:47:02.278618 21212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d555c2ecc0974c01a66aab13c7fe01ca" candidate_uuid: "b743f853697c4862a645b0f2daf59c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ccd5f3595f420f9c941864a5c0f5db"
I20260807 19:47:02.278697 21361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d555c2ecc0974c01a66aab13c7fe01ca" candidate_uuid: "b743f853697c4862a645b0f2daf59c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2268017eaed418180fbdb46965e83db"
I20260807 19:47:02.278803 21212 raft_consensus.cc:3037] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:02.278899 21361 raft_consensus.cc:3037] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:02.281037 21212 raft_consensus.cc:2454] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b743f853697c4862a645b0f2daf59c88 in term 1.
I20260807 19:47:02.281108 21361 raft_consensus.cc:2454] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b743f853697c4862a645b0f2daf59c88 in term 1.
I20260807 19:47:02.281432 21430 leader_election.cc:302] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ccd5f3595f420f9c941864a5c0f5db, b743f853697c4862a645b0f2daf59c88; no voters:
I20260807 19:47:02.281708 21593 raft_consensus.cc:2781] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:02.282176 21593 raft_consensus.cc:686] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [term 1 LEADER]: Becoming Leader. State: Replica: b743f853697c4862a645b0f2daf59c88, State: Running, Role: LEADER
I20260807 19:47:02.282652 21593 consensus_queue.cc:237] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ccd5f3595f420f9c941864a5c0f5db" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39287 } } peers { permanent_uuid: "b743f853697c4862a645b0f2daf59c88" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 45779 } } peers { permanent_uuid: "b2268017eaed418180fbdb46965e83db" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34465 } }
I20260807 19:47:02.287401 21057 catalog_manager.cc:5495] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 reported cstate change: term changed from 0 to 1, leader changed from <none> to b743f853697c4862a645b0f2daf59c88 (127.30.208.3). New cstate: current_term: 1 leader_uuid: "b743f853697c4862a645b0f2daf59c88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ccd5f3595f420f9c941864a5c0f5db" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b743f853697c4862a645b0f2daf59c88" member_type: VOTER last_known_addr { host: "127.30.208.3" port: 45779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2268017eaed418180fbdb46965e83db" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34465 } health_report { overall_health: UNKNOWN } } }
W20260807 19:47:02.290707 21563 tablet.cc:2369] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:47:02.312922 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.307922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53202 (local address 127.30.208.3:45779)
0807 19:47:02.308115 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:02.308120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:02.308138 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:47:02.308225 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:02.308229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:02.308315 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:02.308443 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:02.308801 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.309628 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.311023 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.311439 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.311533 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.311924 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:02.312451 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:02.312558 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:02.312661 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 19:47:02.312726 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:47:02.321152 21212 raft_consensus.cc:1261] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db [term 1 FOLLOWER]: Refusing update from remote peer b743f853697c4862a645b0f2daf59c88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:02.321152 21361 raft_consensus.cc:1261] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db [term 1 FOLLOWER]: Refusing update from remote peer b743f853697c4862a645b0f2daf59c88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:02.322180 21595 consensus_queue.cc:1035] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2268017eaed418180fbdb46965e83db" member_type: VOTER last_known_addr { host: "127.30.208.2" port: 34465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:02.322563 21593 consensus_queue.cc:1035] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ccd5f3595f420f9c941864a5c0f5db" member_type: VOTER last_known_addr { host: "127.30.208.1" port: 39287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 19:47:02.331791 21410 tablet.cc:2369] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:47:02.334705 21600 mvcc.cc:205] Tried to move back new op lower bound from 7315996763416879104 to 7315996763273641984. Current Snapshot: MvccSnapshot[applied={T|T < 7315996763416879104}]
I20260807 19:47:02.352660 21057 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:47:02.352946 21057 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 19:47:02.356830 21057 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 326392304e804ad689e15da87d655d5a: Sending DeleteTablet for 3 replicas of tablet d555c2ecc0974c01a66aab13c7fe01ca
I20260807 19:47:02.358176 21486 tablet_service.cc:1514] Processing DeleteTablet for tablet d555c2ecc0974c01a66aab13c7fe01ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:02 UTC) from {username='slave'} at 127.0.0.1:53200
I20260807 19:47:02.358220 21337 tablet_service.cc:1514] Processing DeleteTablet for tablet d555c2ecc0974c01a66aab13c7fe01ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:02 UTC) from {username='slave'} at 127.0.0.1:58756
I20260807 19:47:02.359205 21192 tablet_service.cc:1514] Processing DeleteTablet for tablet d555c2ecc0974c01a66aab13c7fe01ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 19:47:02 UTC) from {username='slave'} at 127.0.0.1:44762
I20260807 19:47:02.359623 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 21122
I20260807 19:47:02.386021 21611 tablet_replica.cc:330] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db: stopping tablet replica
I20260807 19:47:02.386610 21611 raft_consensus.cc:2229] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:47:02.387223 21611 raft_consensus.cc:2258] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:47:02.390450 21578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.361070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44778 (local address 127.30.208.1:39287)
0807 19:47:02.361278 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:02.361284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:02.380137 (+ 18853us) server_negotiation.cc:407] Connection header received
0807 19:47:02.380717 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:02.380725 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:02.380854 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:02.380938 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:02.381730 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.382927 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.384981 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.385817 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.385927 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.388472 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:02.388486 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:02.388506 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:02.389409 (+ 903us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:02.389479 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:02.389508 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:02.389601 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:02.389918 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 19:47:02.390051 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 19:47:02.390779 21611 ts_tablet_manager.cc:1905] T d555c2ecc0974c01a66aab13c7fe01ca P 22ccd5f3595f420f9c941864a5c0f5db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:02.391598 21609 tablet_replica.cc:330] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db: stopping tablet replica
I20260807 19:47:02.391728 21254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:02.392642 21610 tablet_replica.cc:330] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88: stopping tablet replica
I20260807 19:47:02.393086 21610 raft_consensus.cc:2229] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [term 1 LEADER]: Raft consensus shutting down.
I20260807 19:47:02.394375 21609 raft_consensus.cc:2229] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 19:47:02.394985 21609 raft_consensus.cc:2258] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:47:02.395481 21610 raft_consensus.cc:2258] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 19:47:02.398459 21609 ts_tablet_manager.cc:1905] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:02.401479 21610 ts_tablet_manager.cc:1905] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 19:47:02.406191 21609 ts_tablet_manager.cc:1918] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:47:02.406399 21609 log.cc:1196] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-1/wal/wals/d555c2ecc0974c01a66aab13c7fe01ca
I20260807 19:47:02.406971 21609 ts_tablet_manager.cc:1939] T d555c2ecc0974c01a66aab13c7fe01ca P b2268017eaed418180fbdb46965e83db: Deleting consensus metadata
I20260807 19:47:02.409096 21041 catalog_manager.cc:4841] TS b2268017eaed418180fbdb46965e83db (127.30.208.2:34465): tablet d555c2ecc0974c01a66aab13c7fe01ca (table test-table [id=61edba72a98b438ca37bbb5f825c13ab]) successfully deleted
I20260807 19:47:02.410789 21610 ts_tablet_manager.cc:1918] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 19:47:02.411486 21610 log.cc:1196] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88: Deleting WAL directory at /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786131702042267-31552-0/minicluster-data/ts-2/wal/wals/d555c2ecc0974c01a66aab13c7fe01ca
I20260807 19:47:02.412510 21610 ts_tablet_manager.cc:1939] T d555c2ecc0974c01a66aab13c7fe01ca P b743f853697c4862a645b0f2daf59c88: Deleting consensus metadata
I20260807 19:47:02.414639 21040 catalog_manager.cc:4841] TS b743f853697c4862a645b0f2daf59c88 (127.30.208.3:45779): tablet d555c2ecc0974c01a66aab13c7fe01ca (table test-table [id=61edba72a98b438ca37bbb5f825c13ab]) successfully deleted
I20260807 19:47:02.509461 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 21122
W20260807 19:47:02.534201 21041 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.208.1:39287 (error 108)
I20260807 19:47:02.534488 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 21266
I20260807 19:47:02.543177 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.542587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39287 (local address 127.0.0.1:44794)
0807 19:47:02.542832 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:02.542961 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 19:47:02.543552 21041 catalog_manager.cc:4584] TS 22ccd5f3595f420f9c941864a5c0f5db (127.30.208.1:39287): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d555c2ecc0974c01a66aab13c7fe01ca: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39287: connect: Connection refused (error 111)
I20260807 19:47:02.546644 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.535248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58764 (local address 127.30.208.2:34465)
0807 19:47:02.535434 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:02.535440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:02.535531 (+ 91us) server_negotiation.cc:407] Connection header received
0807 19:47:02.535713 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:02.535718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:02.535839 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:02.535973 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:02.536450 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.537552 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.539459 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.540091 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.540212 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.544948 (+ 4736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:02.544960 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:02.544978 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:02.545815 (+ 837us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:02.545887 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:02.545914 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:02.546035 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:02.546174 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 19:47:02.546300 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 19:47:02.547520 21404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:02.597360 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.596719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39287 (local address 127.0.0.1:44804)
0807 19:47:02.596958 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:02.597105 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 19:47:02.598627 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.598185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39287 (local address 127.0.0.1:44806)
0807 19:47:02.598356 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:02.598450 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 19:47:02.638893 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 21266
I20260807 19:47:02.656318 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 21412
I20260807 19:47:02.659485 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.658965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39287 (local address 127.0.0.1:44820)
0807 19:47:02.659202 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:02.659310 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 19:47:02.660643 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.660246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39287 (local address 127.0.0.1:44836)
0807 19:47:02.660412 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:02.660502 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 19:47:02.664362 21580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.657519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53208 (local address 127.30.208.3:45779)
0807 19:47:02.657699 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:02.657703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:02.657719 (+ 16us) server_negotiation.cc:407] Connection header received
0807 19:47:02.657779 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:02.657782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:02.657859 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:02.657950 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:02.658403 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.659501 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.661147 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:02.661698 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:02.661822 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:02.662974 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:02.662984 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:02.662998 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:02.663619 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:02.663666 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:02.663688 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:02.663780 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:02.663930 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 19:47:02.664033 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 19:47:02.665025 21556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:02.741441 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.740884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39287 (local address 127.0.0.1:44838)
0807 19:47:02.741127 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:02.741263 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 19:47:02.742619 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.742169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39287 (local address 127.0.0.1:44840)
0807 19:47:02.742379 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:02.742478 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 19:47:02.788144 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 21412
I20260807 19:47:02.808702 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 21023
I20260807 19:47:02.809954 21090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:02.918754 21577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:02.917848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.1:39287 (local address 127.0.0.1:44852)
0807 19:47:02.918042 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:02.918152 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.1:39287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 19:47:02.918763 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 21023
2026-08-07T19:47:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2902 ms)
[----------] 6 tests from EncryptionPolicyTest (21600 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:03 dist-test-slave-vq2q krb5kdc[21624](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:03 dist-test-slave-vq2q krb5kdc[21624](info): set up 2 sockets
Aug 07 19:47:03 dist-test-slave-vq2q krb5kdc[21624](info): commencing operation
krb5kdc: starting...
W20260807 19:47:04.024230 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:04 dist-test-slave-vq2q krb5kdc[21624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:47:04.329259 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37667
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:04.667150 21638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:04.667483 21638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:04.667620 21638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:04.675550 21638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:04.675690 21638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:04.675770 21638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:04.675863 21638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:04.675925 21638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:04.686080 21638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37667
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:04.687516 21638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:04.688863 21638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:04.700476 21645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:04.700685 21647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:04.700520 21644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:04.701546 21638 server_base.cc:1034] running on GCE node
I20260807 19:47:04.702304 21638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:04.703810 21638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:04.705040 21638 hybrid_clock.cc:648] HybridClock initialized: now 1786132024704987 us; error 35 us; skew 500 ppm
I20260807 19:47:04.705435 21638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:04 dist-test-slave-vq2q krb5kdc[21624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:04.708662 21638 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:47:04.710280 21638 webserver.cc:460] Webserver started at http://127.0.0.1:33903/ using document root <none> and password file <none>
I20260807 19:47:04.710927 21638 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:04.711030 21638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:04.711298 21638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:04.714366 21638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "af240db7086849f9a7251e5c03b3b98e"
format_stamp: "Formatted at 2026-08-07 19:47:04 on dist-test-slave-vq2q"
server_key: "93c91af12d14e065910ab8fbf36e99f4"
server_key_iv: "c4232d8ac4d8ac9d3e49da922bf5078f"
server_key_version: "encryptionkey@0"
I20260807 19:47:04.715103 21638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "af240db7086849f9a7251e5c03b3b98e"
format_stamp: "Formatted at 2026-08-07 19:47:04 on dist-test-slave-vq2q"
server_key: "93c91af12d14e065910ab8fbf36e99f4"
server_key_iv: "c4232d8ac4d8ac9d3e49da922bf5078f"
server_key_version: "encryptionkey@0"
I20260807 19:47:04.720197 21638 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:47:04.723723 21654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:04.725119 21638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 19:47:04.725342 21638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "af240db7086849f9a7251e5c03b3b98e"
format_stamp: "Formatted at 2026-08-07 19:47:04 on dist-test-slave-vq2q"
server_key: "93c91af12d14e065910ab8fbf36e99f4"
server_key_iv: "c4232d8ac4d8ac9d3e49da922bf5078f"
server_key_version: "encryptionkey@0"
I20260807 19:47:04.725526 21638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:04.736963 21638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:04.741150 21638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:04.741458 21638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:04.758855 21638 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37667
I20260807 19:47:04.758859 21706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37667 every 8 connection(s)
I20260807 19:47:04.760456 21638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:04.764475 21707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:04.769287 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 21638
I20260807 19:47:04.769491 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:04.770130 31552 external_mini_cluster.cc:1428] Setting key b9e330db073eca4fbb2092d1d944b3de
I20260807 19:47:04.776270 21707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e: Bootstrap starting.
I20260807 19:47:04.780108 21707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:04.781430 21707 log.cc:824] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:04.785126 21707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e: No bootstrap required, opened a new log
Aug 07 19:47:04 dist-test-slave-vq2q krb5kdc[21624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:47:04.795521 21707 raft_consensus.cc:348] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af240db7086849f9a7251e5c03b3b98e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37667 } }
I20260807 19:47:04.796007 21707 raft_consensus.cc:374] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:04.796162 21711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:04.772054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45527 (local address 127.0.0.1:37667)
0807 19:47:04.772916 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:04.772928 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:04.772970 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:47:04.774883 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:04.774910 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:04.775334 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:04.775673 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:04.777623 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:04.779052 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:04.780828 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:04.781633 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:04.781829 (+ 196us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:04.786700 (+ 4871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:04.786713 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:04.786733 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:04.786785 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:04.791008 (+ 4223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:04.791955 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:04.791965 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:04.791978 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:04.792201 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:04.792750 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:04.792755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:04.792759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:04.793592 (+ 833us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:04.793926 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:04.794401 (+ 475us) server_negotiation.cc:299] Negotiation successful
0807 19:47:04.794771 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":404,"threads_started":1}
I20260807 19:47:04.796197 21707 raft_consensus.cc:729] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af240db7086849f9a7251e5c03b3b98e, State: Initialized, Role: FOLLOWER
I20260807 19:47:04.796890 21707 consensus_queue.cc:260] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af240db7086849f9a7251e5c03b3b98e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37667 } }
I20260807 19:47:04.797137 21707 raft_consensus.cc:388] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:04.797305 21707 raft_consensus.cc:482] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:04.797477 21707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:04.800482 21707 raft_consensus.cc:504] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af240db7086849f9a7251e5c03b3b98e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37667 } }
I20260807 19:47:04.800647 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:04.801085 21707 leader_election.cc:302] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af240db7086849f9a7251e5c03b3b98e; no voters:
I20260807 19:47:04.801864 21707 leader_election.cc:288] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:04.801990 21713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:04.803119 21713 raft_consensus.cc:686] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [term 1 LEADER]: Becoming Leader. State: Replica: af240db7086849f9a7251e5c03b3b98e, State: Running, Role: LEADER
I20260807 19:47:04.803664 21713 consensus_queue.cc:237] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af240db7086849f9a7251e5c03b3b98e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37667 } }
I20260807 19:47:04.804168 21707 sys_catalog.cc:564] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:04.810461 21715 sys_catalog.cc:455] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [sys.catalog]: SysCatalogTable state changed. Reason: New leader af240db7086849f9a7251e5c03b3b98e. Latest consensus state: current_term: 1 leader_uuid: "af240db7086849f9a7251e5c03b3b98e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af240db7086849f9a7251e5c03b3b98e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37667 } } }
I20260807 19:47:04.810849 21715 sys_catalog.cc:458] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:04.811324 21714 sys_catalog.cc:455] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af240db7086849f9a7251e5c03b3b98e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af240db7086849f9a7251e5c03b3b98e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37667 } } }
I20260807 19:47:04.811635 21714 sys_catalog.cc:458] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:04.821888 21727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:04.826792 21711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:04.806841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51293 (local address 127.0.0.1:37667)
0807 19:47:04.807508 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:04.807514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:04.807542 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:47:04.807656 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:04.807662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:04.807746 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:04.807858 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:04.810042 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:04.811001 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:04.812958 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:04.813723 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:04.813846 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:04.817725 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:04.817734 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:04.817740 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:04.817788 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:04.821882 (+ 4094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:04.822994 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:04.823001 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:04.823005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:04.823175 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:04.823720 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:04.823725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:04.823729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:04.824258 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:04.825889 (+ 1631us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:04.826161 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 19:47:04.826393 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495}
I20260807 19:47:04.827442 21727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:04.837488 21727 catalog_manager.cc:1347] Generated new cluster ID: a81bdb8caec2481d973e0b4a59511fb4
I20260807 19:47:04.837682 21727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:04.850499 21727 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:04.851801 21727 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:04.859730 21727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af240db7086849f9a7251e5c03b3b98e: Generated new TSK 0
I20260807 19:47:04.860596 21727 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:04.867136 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 21638
I20260807 19:47:04.868607 21702 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:04.958504 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 21638
W20260807 19:47:04.975970 21621 connection.cc:535] client connection to 127.0.0.1:37667 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37667): Connection reset by peer (error 104)
2026-08-07T19:47:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2034 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:05 dist-test-slave-vq2q krb5kdc[21741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:05 dist-test-slave-vq2q krb5kdc[21741](info): set up 2 sockets
Aug 07 19:47:05 dist-test-slave-vq2q krb5kdc[21741](info): commencing operation
krb5kdc: starting...
W20260807 19:47:06.052304 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:06 dist-test-slave-vq2q krb5kdc[21741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:47:06.354187 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:06.688601 21754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:06.688931 21754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:06.689069 21754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:06.696538 21754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:06.696676 21754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:06.696761 21754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:06.696849 21754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:06.696904 21754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:06.707226 21754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38607
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:06.708593 21754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:06.709959 21754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:06.722373 21759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:06.722311 21762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:06.722407 21760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:06.723454 21754 server_base.cc:1034] running on GCE node
I20260807 19:47:06.724282 21754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:06.725795 21754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:06.727013 21754 hybrid_clock.cc:648] HybridClock initialized: now 1786132026726967 us; error 37 us; skew 500 ppm
I20260807 19:47:06.727367 21754 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:06 dist-test-slave-vq2q krb5kdc[21741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132026, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:06.730564 21754 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:47:06.732177 21754 webserver.cc:460] Webserver started at http://127.0.0.1:46345/ using document root <none> and password file <none>
I20260807 19:47:06.732864 21754 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:06.732985 21754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:06.733325 21754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:06.736371 21754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "dfa5c8b7989047bf9ed57d0b775d09ba"
format_stamp: "Formatted at 2026-08-07 19:47:06 on dist-test-slave-vq2q"
server_key: "94a68d78418a857c71e6a08263b685b6"
server_key_iv: "2229fb433c02f0432daaaf3626a253cf"
server_key_version: "encryptionkey@0"
I20260807 19:47:06.737128 21754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "dfa5c8b7989047bf9ed57d0b775d09ba"
format_stamp: "Formatted at 2026-08-07 19:47:06 on dist-test-slave-vq2q"
server_key: "94a68d78418a857c71e6a08263b685b6"
server_key_iv: "2229fb433c02f0432daaaf3626a253cf"
server_key_version: "encryptionkey@0"
I20260807 19:47:06.742326 21754 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 19:47:06.746011 21772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:06.747352 21754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:47:06.747545 21754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "dfa5c8b7989047bf9ed57d0b775d09ba"
format_stamp: "Formatted at 2026-08-07 19:47:06 on dist-test-slave-vq2q"
server_key: "94a68d78418a857c71e6a08263b685b6"
server_key_iv: "2229fb433c02f0432daaaf3626a253cf"
server_key_version: "encryptionkey@0"
I20260807 19:47:06.747745 21754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:06.766924 21754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:06.771505 21754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:06.771883 21754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:06.790195 21754 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38607
I20260807 19:47:06.790231 21829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38607 every 8 connection(s)
I20260807 19:47:06.791603 21754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:06.793123 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 21754
I20260807 19:47:06.793329 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:06.793896 31552 external_mini_cluster.cc:1428] Setting key be8ca7526ba0af565bcc8aa8499caf9c
I20260807 19:47:06.796065 21830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:06.807816 21833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:06.795669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59931 (local address 127.0.0.1:38607)
0807 19:47:06.796447 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:06.796460 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:06.796502 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:47:06.798174 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:06.798196 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:06.798595 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:06.798951 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:06.799601 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:06.800974 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:06.802874 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:06.803566 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:06.803763 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:06.805263 (+ 1500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:06.805275 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:06.805293 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:06.805911 (+ 618us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:06.805994 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:06.806026 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:06.806153 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:06.806507 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 19:47:06.806727 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":362,"threads_started":1}
I20260807 19:47:06.808794 21830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba: Bootstrap starting.
I20260807 19:47:06.811676 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:06.812063 21830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:06.813030 21830 log.cc:824] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:06.816084 21830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba: No bootstrap required, opened a new log
I20260807 19:47:06.822291 21833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:06.816122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46695 (local address 127.0.0.1:38607)
0807 19:47:06.816292 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:06.816298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:06.816720 (+ 422us) server_negotiation.cc:407] Connection header received
0807 19:47:06.816914 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:06.816920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:06.816987 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:06.817069 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:06.817662 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:06.818483 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:06.819806 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:06.820253 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:06.820388 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:06.821527 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:06.821540 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:06.821544 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:06.821563 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:06.821605 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:06.821620 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:06.821720 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:06.821936 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 19:47:06.821996 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:47:06.824515 21830 raft_consensus.cc:348] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfa5c8b7989047bf9ed57d0b775d09ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } }
I20260807 19:47:06.824872 21830 raft_consensus.cc:374] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:06.824946 21830 raft_consensus.cc:729] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfa5c8b7989047bf9ed57d0b775d09ba, State: Initialized, Role: FOLLOWER
I20260807 19:47:06.825512 21830 consensus_queue.cc:260] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfa5c8b7989047bf9ed57d0b775d09ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } }
I20260807 19:47:06.825752 21830 raft_consensus.cc:388] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:06.825855 21830 raft_consensus.cc:482] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:06.825981 21830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:06.828130 21830 raft_consensus.cc:504] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfa5c8b7989047bf9ed57d0b775d09ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } }
I20260807 19:47:06.828647 21830 leader_election.cc:302] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dfa5c8b7989047bf9ed57d0b775d09ba; no voters:
I20260807 19:47:06.829303 21830 leader_election.cc:288] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:06.829350 21841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:06.831446 21830 sys_catalog.cc:564] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:06.832914 21841 raft_consensus.cc:686] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [term 1 LEADER]: Becoming Leader. State: Replica: dfa5c8b7989047bf9ed57d0b775d09ba, State: Running, Role: LEADER
I20260807 19:47:06.833561 21841 consensus_queue.cc:237] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfa5c8b7989047bf9ed57d0b775d09ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } }
I20260807 19:47:06.836537 21843 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader dfa5c8b7989047bf9ed57d0b775d09ba. Latest consensus state: current_term: 1 leader_uuid: "dfa5c8b7989047bf9ed57d0b775d09ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfa5c8b7989047bf9ed57d0b775d09ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } } }
I20260807 19:47:06.836971 21843 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:06.836556 21842 sys_catalog.cc:455] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dfa5c8b7989047bf9ed57d0b775d09ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dfa5c8b7989047bf9ed57d0b775d09ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } } }
I20260807 19:47:06.837364 21842 sys_catalog.cc:458] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:06.838586 21853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:06.843379 21853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:06.851378 21853 catalog_manager.cc:1347] Generated new cluster ID: 41e9a2c1f7fc4a7c97a4b09bd0a4a5ee
I20260807 19:47:06.851498 21853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:06.869835 21853 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:06.871588 21853 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:06.885079 21853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dfa5c8b7989047bf9ed57d0b775d09ba: Generated new TSK 0
I20260807 19:47:06.885965 21853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:06.897748 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 21754
W20260807 19:47:06.899346 21823 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59931
W20260807 19:47:06.900070 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 21754: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:47:06.900269 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 21754
W20260807 19:47:06.917346 21739 connection.cc:535] client connection to 127.0.0.1:38607 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38607): Connection reset by peer (error 104)
2026-08-07T19:47:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1941 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:07 dist-test-slave-vq2q krb5kdc[21865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:07 dist-test-slave-vq2q krb5kdc[21865](info): set up 2 sockets
Aug 07 19:47:07 dist-test-slave-vq2q krb5kdc[21865](info): commencing operation
krb5kdc: starting...
W20260807 19:47:07.995388 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.002s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:08 dist-test-slave-vq2q krb5kdc[21865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:47:08.296074 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41933
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:08.633139 21882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:08.633491 21882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:08.633625 21882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:08.640920 21882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:08.641027 21882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:08.641103 21882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:08.641211 21882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:08.641275 21882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:08.651211 21882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41933
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:08.652508 21882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:08.653918 21882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:08.665975 21888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:08.666050 21890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:08.666080 21887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:08.667143 21882 server_base.cc:1034] running on GCE node
I20260807 19:47:08.667930 21882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:08.669430 21882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:08.670639 21882 hybrid_clock.cc:648] HybridClock initialized: now 1786132028670593 us; error 47 us; skew 500 ppm
I20260807 19:47:08.670974 21882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:08 dist-test-slave-vq2q krb5kdc[21865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132028, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:08.678838 21882 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:47:08.680359 21882 webserver.cc:460] Webserver started at http://127.0.0.1:44155/ using document root <none> and password file <none>
I20260807 19:47:08.681068 21882 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:08.681236 21882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:08.681561 21882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:08.684662 21882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "9ca2d0d9c0144189b99d5876fd60a614"
format_stamp: "Formatted at 2026-08-07 19:47:08 on dist-test-slave-vq2q"
server_key: "17e143e73f8d92aea9a2af2bf1fc28b7"
server_key_iv: "69bdbeb9a91aad6a15cb3bdf76c71f6a"
server_key_version: "encryptionkey@0"
I20260807 19:47:08.685515 21882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "9ca2d0d9c0144189b99d5876fd60a614"
format_stamp: "Formatted at 2026-08-07 19:47:08 on dist-test-slave-vq2q"
server_key: "17e143e73f8d92aea9a2af2bf1fc28b7"
server_key_iv: "69bdbeb9a91aad6a15cb3bdf76c71f6a"
server_key_version: "encryptionkey@0"
I20260807 19:47:08.690804 21882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 19:47:08.694370 21897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:08.695652 21882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:47:08.695822 21882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "9ca2d0d9c0144189b99d5876fd60a614"
format_stamp: "Formatted at 2026-08-07 19:47:08 on dist-test-slave-vq2q"
server_key: "17e143e73f8d92aea9a2af2bf1fc28b7"
server_key_iv: "69bdbeb9a91aad6a15cb3bdf76c71f6a"
server_key_version: "encryptionkey@0"
I20260807 19:47:08.696008 21882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:08.701876 21882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:08.705860 21882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:08.706152 21882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:08.723397 21882 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41933
I20260807 19:47:08.723435 21953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41933 every 8 connection(s)
I20260807 19:47:08.724735 21882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:08.725061 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 21882
I20260807 19:47:08.725229 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:08.725807 31552 external_mini_cluster.cc:1428] Setting key 3dcb69cd15a7b88483888501dbd6029d
I20260807 19:47:08.728991 21954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:08.733492 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:08.727552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49049 (local address 127.0.0.1:41933)
0807 19:47:08.728302 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:08.728313 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:08.728354 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:47:08.730015 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:08.730036 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:08.730412 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:08.730800 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:08.731875 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:08.731884 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:08.731896 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:08.731910 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:08.731991 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:08.732013 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:08.732248 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:08.732376 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 19:47:08.732569 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":336,"threads_started":1}
I20260807 19:47:08.736856 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:08.739985 21954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614: Bootstrap starting.
I20260807 19:47:08.743400 21957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:08.741103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33305 (local address 127.0.0.1:41933)
0807 19:47:08.741337 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:08.741343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:08.741654 (+ 311us) server_negotiation.cc:407] Connection header received
0807 19:47:08.741835 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:08.741838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:08.741885 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:08.741958 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:08.742840 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:08.742843 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:08.742846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:08.742860 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:08.742896 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:08.742911 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:08.742991 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:08.743105 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 19:47:08.743147 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260807 19:47:08.743623 21954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:08.744735 21954 log.cc:824] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:08.747498 21954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614: No bootstrap required, opened a new log
I20260807 19:47:08.754900 21954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca2d0d9c0144189b99d5876fd60a614" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41933 } }
I20260807 19:47:08.755287 21954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:08.755378 21954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ca2d0d9c0144189b99d5876fd60a614, State: Initialized, Role: FOLLOWER
I20260807 19:47:08.755906 21954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca2d0d9c0144189b99d5876fd60a614" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41933 } }
I20260807 19:47:08.756117 21954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:08.756250 21954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:08.756403 21954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:08.758666 21954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca2d0d9c0144189b99d5876fd60a614" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41933 } }
I20260807 19:47:08.759146 21954 leader_election.cc:302] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ca2d0d9c0144189b99d5876fd60a614; no voters:
I20260807 19:47:08.759913 21954 leader_election.cc:288] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:08.760002 21966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:08.760705 21966 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [term 1 LEADER]: Becoming Leader. State: Replica: 9ca2d0d9c0144189b99d5876fd60a614, State: Running, Role: LEADER
I20260807 19:47:08.761291 21966 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca2d0d9c0144189b99d5876fd60a614" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41933 } }
I20260807 19:47:08.762040 21954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:08.764153 21968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ca2d0d9c0144189b99d5876fd60a614. Latest consensus state: current_term: 1 leader_uuid: "9ca2d0d9c0144189b99d5876fd60a614" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca2d0d9c0144189b99d5876fd60a614" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41933 } } }
I20260807 19:47:08.764617 21968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:08.764246 21967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ca2d0d9c0144189b99d5876fd60a614" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ca2d0d9c0144189b99d5876fd60a614" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41933 } } }
I20260807 19:47:08.764992 21967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614 [sys.catalog]: This master's current role is: LEADER
W20260807 19:47:08.772789 21986 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:47:08.772909 21986 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:47:08.773049 21977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:08.774807 21977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:08.782934 21977 catalog_manager.cc:1347] Generated new cluster ID: dcd4fab162f24a1e951d5148b5062998
I20260807 19:47:08.783059 21977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:08.818351 21977 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:08.818769 21977 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:08.830713 21977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ca2d0d9c0144189b99d5876fd60a614: Generated new TSK 0
I20260807 19:47:08.831542 21977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:08.842393 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 21882
W20260807 19:47:08.843524 21948 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49049
W20260807 19:47:08.844261 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 21882: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:47:08.844403 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 21882
2026-08-07T19:47:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1945 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:08 dist-test-slave-vq2q krb5kdc[21991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:08 dist-test-slave-vq2q krb5kdc[21991](info): set up 2 sockets
Aug 07 19:47:08 dist-test-slave-vq2q krb5kdc[21991](info): commencing operation
krb5kdc: starting...
W20260807 19:47:09.944762 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:10 dist-test-slave-vq2q krb5kdc[21991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:47:10.250577 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41837
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41837
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:10.589382 22004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:10.589691 22004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:10.589841 22004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:10.597141 22004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:10.597287 22004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:10.597379 22004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:10.597472 22004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:10.597536 22004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:10.607354 22004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:10.608609 22004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:10.609898 22004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:10.621213 22010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:10.621165 22011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:10.621240 22013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:10.621907 22004 server_base.cc:1034] running on GCE node
I20260807 19:47:10.622649 22004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:10.624001 22004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:10.625227 22004 hybrid_clock.cc:648] HybridClock initialized: now 1786132030625148 us; error 48 us; skew 500 ppm
I20260807 19:47:10.625566 22004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:10 dist-test-slave-vq2q krb5kdc[21991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132030, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:10.628574 22004 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:47:10.630106 22004 webserver.cc:460] Webserver started at http://127.0.0.1:33587/ using document root <none> and password file <none>
I20260807 19:47:10.630777 22004 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:10.630899 22004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:10.631189 22004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:10.634250 22004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "7ce79a8428cb436a8e0a58ad0fc9060f"
format_stamp: "Formatted at 2026-08-07 19:47:10 on dist-test-slave-vq2q"
server_key: "58e95ef5fea6fad70254afe57732f9e0"
server_key_iv: "89a02a73bda192b77e49720d53e2b536"
server_key_version: "encryptionkey@0"
I20260807 19:47:10.635027 22004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "7ce79a8428cb436a8e0a58ad0fc9060f"
format_stamp: "Formatted at 2026-08-07 19:47:10 on dist-test-slave-vq2q"
server_key: "58e95ef5fea6fad70254afe57732f9e0"
server_key_iv: "89a02a73bda192b77e49720d53e2b536"
server_key_version: "encryptionkey@0"
I20260807 19:47:10.640096 22004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 19:47:10.643482 22020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:10.644771 22004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 19:47:10.644934 22004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "7ce79a8428cb436a8e0a58ad0fc9060f"
format_stamp: "Formatted at 2026-08-07 19:47:10 on dist-test-slave-vq2q"
server_key: "58e95ef5fea6fad70254afe57732f9e0"
server_key_iv: "89a02a73bda192b77e49720d53e2b536"
server_key_version: "encryptionkey@0"
I20260807 19:47:10.645116 22004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:10.659235 22004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:10.663264 22004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:10.663525 22004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:10.681159 22004 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41837
I20260807 19:47:10.681255 22081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41837 every 8 connection(s)
I20260807 19:47:10.682554 22004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:10.686400 22082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:10.689627 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 22004
I20260807 19:47:10.689844 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:10.690371 31552 external_mini_cluster.cc:1428] Setting key 72c374dfd48cd0fd287e85cf5d18d3ca
I20260807 19:47:10.698879 22082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f: Bootstrap starting.
I20260807 19:47:10.702740 22082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:10.704084 22082 log.cc:824] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:47:10 dist-test-slave-vq2q krb5kdc[21991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:47:10.707505 22082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f: No bootstrap required, opened a new log
I20260807 19:47:10.713965 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:10.692270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45779 (local address 127.0.0.1:41837)
0807 19:47:10.693051 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:10.693068 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:10.693110 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:47:10.694890 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:10.694914 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:10.695328 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:10.695646 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:10.697478 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:10.698783 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:10.700497 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:10.701172 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:10.701360 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:10.706599 (+ 5239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:10.706608 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:10.706626 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:10.706664 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:10.709694 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:10.710666 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:10.710673 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:10.710679 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:10.710859 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:10.711278 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:10.711281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:10.711286 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:10.711805 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:10.712027 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:10.712505 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 19:47:10.712879 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":360,"threads_started":1}
I20260807 19:47:10.716593 22082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce79a8428cb436a8e0a58ad0fc9060f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41837 } }
I20260807 19:47:10.716940 22082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:10.717036 22082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ce79a8428cb436a8e0a58ad0fc9060f, State: Initialized, Role: FOLLOWER
I20260807 19:47:10.717576 22082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce79a8428cb436a8e0a58ad0fc9060f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41837 } }
I20260807 19:47:10.717710 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:10.717803 22082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:10.717931 22082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:10.718086 22082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:10.720373 22082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce79a8428cb436a8e0a58ad0fc9060f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41837 } }
I20260807 19:47:10.720944 22082 leader_election.cc:302] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ce79a8428cb436a8e0a58ad0fc9060f; no voters:
I20260807 19:47:10.721587 22082 leader_election.cc:288] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:10.721704 22092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:10.723543 22092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [term 1 LEADER]: Becoming Leader. State: Replica: 7ce79a8428cb436a8e0a58ad0fc9060f, State: Running, Role: LEADER
I20260807 19:47:10.723877 22082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:10.724180 22092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce79a8428cb436a8e0a58ad0fc9060f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41837 } }
I20260807 19:47:10.730757 22095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ce79a8428cb436a8e0a58ad0fc9060f. Latest consensus state: current_term: 1 leader_uuid: "7ce79a8428cb436a8e0a58ad0fc9060f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce79a8428cb436a8e0a58ad0fc9060f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41837 } } }
I20260807 19:47:10.731201 22095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:10.730813 22093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ce79a8428cb436a8e0a58ad0fc9060f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ce79a8428cb436a8e0a58ad0fc9060f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41837 } } }
I20260807 19:47:10.731599 22093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:10.733273 22104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:10.739022 22104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:10.740700 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:10.722131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42699 (local address 127.0.0.1:41837)
0807 19:47:10.722318 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:10.722324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:10.723155 (+ 831us) server_negotiation.cc:407] Connection header received
0807 19:47:10.723332 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:10.723337 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:10.723427 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:10.723544 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:10.725262 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:10.726313 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:10.727607 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:10.728051 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:10.728157 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:10.733227 (+ 5070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:10.733234 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:10.733240 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:10.733280 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:10.736529 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:10.737464 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:10.737470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:10.737474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:10.737631 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:10.738103 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:10.738108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:10.738111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:10.738522 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:10.739937 (+ 1415us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:10.740107 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 19:47:10.740335 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:47:10.748914 22104 catalog_manager.cc:1347] Generated new cluster ID: 6777eeb28b18410c8ce90c4ab282dced
I20260807 19:47:10.749029 22104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:10.763270 22104 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:10.764608 22104 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:10.772253 22104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ce79a8428cb436a8e0a58ad0fc9060f: Generated new TSK 0
I20260807 19:47:10.773141 22104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:10.784016 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 22004
I20260807 19:47:10.785079 22077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:10.875636 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 22004
2026-08-07T19:47:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2030 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:10 dist-test-slave-vq2q krb5kdc[22118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:10 dist-test-slave-vq2q krb5kdc[22118](info): set up 2 sockets
Aug 07 19:47:10 dist-test-slave-vq2q krb5kdc[22118](info): commencing operation
krb5kdc: starting...
W20260807 19:47:11.972321 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:12 dist-test-slave-vq2q krb5kdc[22118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:47:12.274417 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45561
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:12.620764 22135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:12.621063 22135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:12.621165 22135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:12.628557 22135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:12.628674 22135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:12.628728 22135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:12.628794 22135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:12.628839 22135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:12.638854 22135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:12.640127 22135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:12.641418 22135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:12.653154 22140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:12.653088 22143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:12.653127 22141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:12.654083 22135 server_base.cc:1034] running on GCE node
I20260807 19:47:12.654863 22135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:12.656266 22135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:12.657465 22135 hybrid_clock.cc:648] HybridClock initialized: now 1786132032657415 us; error 47 us; skew 500 ppm
I20260807 19:47:12.657788 22135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:12 dist-test-slave-vq2q krb5kdc[22118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132032, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:12.660784 22135 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:47:12.662204 22135 webserver.cc:460] Webserver started at http://127.0.0.1:44653/ using document root <none> and password file <none>
I20260807 19:47:12.662845 22135 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:12.662961 22135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:12.663240 22135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:12.666256 22135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "181600a1d28648328231a77b56383806"
format_stamp: "Formatted at 2026-08-07 19:47:12 on dist-test-slave-vq2q"
server_key: "33e2cc0f8350f39b118bab7ff810fd3c"
server_key_iv: "e2274b89a06f78243ac38c156d28d819"
server_key_version: "encryptionkey@0"
I20260807 19:47:12.667007 22135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "181600a1d28648328231a77b56383806"
format_stamp: "Formatted at 2026-08-07 19:47:12 on dist-test-slave-vq2q"
server_key: "33e2cc0f8350f39b118bab7ff810fd3c"
server_key_iv: "e2274b89a06f78243ac38c156d28d819"
server_key_version: "encryptionkey@0"
I20260807 19:47:12.672009 22135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 19:47:12.675323 22151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:12.676642 22135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 19:47:12.676827 22135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "181600a1d28648328231a77b56383806"
format_stamp: "Formatted at 2026-08-07 19:47:12 on dist-test-slave-vq2q"
server_key: "33e2cc0f8350f39b118bab7ff810fd3c"
server_key_iv: "e2274b89a06f78243ac38c156d28d819"
server_key_version: "encryptionkey@0"
I20260807 19:47:12.677021 22135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:12.724865 22135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:12.729168 22135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:12.729499 22135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:12.747611 22135 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45561
I20260807 19:47:12.747645 22207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45561 every 8 connection(s)
I20260807 19:47:12.748986 22135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:12.752664 22209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:12.754400 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 22135
I20260807 19:47:12.754550 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:12.754977 31552 external_mini_cluster.cc:1428] Setting key 19c8e625a97ad9b13ba18155d23ad716
I20260807 19:47:12.767704 22209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806: Bootstrap starting.
I20260807 19:47:12.770419 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:12.756824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41945 (local address 127.0.0.1:45561)
0807 19:47:12.757679 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:12.757694 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:12.757750 (+ 56us) server_negotiation.cc:407] Connection header received
0807 19:47:12.759418 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:12.759448 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:12.760015 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:12.760491 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:12.761120 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:12.763087 (+ 1967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:12.764843 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:12.766070 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:12.766295 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:12.767499 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:12.767511 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:12.767532 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:12.768273 (+ 741us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:12.768383 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:12.768431 (+ 48us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:12.768539 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:12.768781 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 19:47:12.769061 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":373,"threads_started":1}
I20260807 19:47:12.771540 22209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:12.772862 22209 log.cc:824] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:12.774431 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:12.775831 22209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806: No bootstrap required, opened a new log
I20260807 19:47:12.786162 22209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181600a1d28648328231a77b56383806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45561 } }
I20260807 19:47:12.786763 22209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:12.786852 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:12.778951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40661 (local address 127.0.0.1:45561)
0807 19:47:12.779157 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:12.779165 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:12.779683 (+ 518us) server_negotiation.cc:407] Connection header received
0807 19:47:12.779892 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:12.779896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:12.779964 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:12.780045 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:12.780773 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:12.781734 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:12.783872 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:12.784527 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:12.784675 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:12.785991 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:12.785997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:12.786000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:12.786022 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:12.786070 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:12.786087 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:12.786195 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:12.786423 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 19:47:12.786514 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:47:12.786901 22209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181600a1d28648328231a77b56383806, State: Initialized, Role: FOLLOWER
I20260807 19:47:12.787609 22209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181600a1d28648328231a77b56383806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45561 } }
I20260807 19:47:12.787891 22209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:12.788030 22209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:12.788210 22209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:12.790606 22209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181600a1d28648328231a77b56383806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45561 } }
I20260807 19:47:12.791108 22209 leader_election.cc:302] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 181600a1d28648328231a77b56383806; no voters:
I20260807 19:47:12.791831 22209 leader_election.cc:288] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:12.791971 22220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:12.792611 22220 raft_consensus.cc:686] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [term 1 LEADER]: Becoming Leader. State: Replica: 181600a1d28648328231a77b56383806, State: Running, Role: LEADER
I20260807 19:47:12.793298 22220 consensus_queue.cc:237] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181600a1d28648328231a77b56383806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45561 } }
I20260807 19:47:12.794271 22209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:12.796185 22222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 181600a1d28648328231a77b56383806. Latest consensus state: current_term: 1 leader_uuid: "181600a1d28648328231a77b56383806" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181600a1d28648328231a77b56383806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45561 } } }
I20260807 19:47:12.796226 22221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "181600a1d28648328231a77b56383806" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "181600a1d28648328231a77b56383806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45561 } } }
I20260807 19:47:12.796646 22222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:12.796746 22221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:12.798898 22225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:12.805094 22225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:12.816787 22225 catalog_manager.cc:1347] Generated new cluster ID: 82403d43a4b84a7390db4ff9aea4faf2
I20260807 19:47:12.816938 22225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:12.839771 22225 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:12.841210 22225 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:12.852196 22225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 181600a1d28648328231a77b56383806: Generated new TSK 0
I20260807 19:47:12.853240 22225 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:12.861977 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 22135
W20260807 19:47:12.863557 22202 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41945
W20260807 19:47:12.864298 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 22135: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:47:12.864434 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 22135
2026-08-07T19:47:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1989 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:12 dist-test-slave-vq2q krb5kdc[22245](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:12 dist-test-slave-vq2q krb5kdc[22245](info): set up 2 sockets
Aug 07 19:47:12 dist-test-slave-vq2q krb5kdc[22245](info): commencing operation
krb5kdc: starting...
W20260807 19:47:13.964963 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:14 dist-test-slave-vq2q krb5kdc[22245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:47:14.273545 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38607
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:14.612280 22262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:14.612593 22262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:14.612735 22262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:14.619995 22262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:14.620129 22262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:14.620215 22262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:14.620304 22262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:14.620369 22262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:14.630581 22262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:14.631898 22262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:14.633249 22262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:14.644944 22268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:14.644989 22270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:14.645038 22267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:14.645880 22262 server_base.cc:1034] running on GCE node
I20260807 19:47:14.646739 22262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:14.648209 22262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:14.649430 22262 hybrid_clock.cc:648] HybridClock initialized: now 1786132034649379 us; error 41 us; skew 500 ppm
I20260807 19:47:14.649823 22262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:14 dist-test-slave-vq2q krb5kdc[22245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132034, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:14.656204 22262 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:47:14.657902 22262 webserver.cc:460] Webserver started at http://127.0.0.1:33231/ using document root <none> and password file <none>
I20260807 19:47:14.658588 22262 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:14.658715 22262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:14.659018 22262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:14.662117 22262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "309069a3fc254f638268ed213d48cec3"
format_stamp: "Formatted at 2026-08-07 19:47:14 on dist-test-slave-vq2q"
server_key: "e274781ab8833745e2e75b46628af289"
server_key_iv: "0683bbabe02c297b2a1d562436633cdd"
server_key_version: "encryptionkey@0"
I20260807 19:47:14.662894 22262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "309069a3fc254f638268ed213d48cec3"
format_stamp: "Formatted at 2026-08-07 19:47:14 on dist-test-slave-vq2q"
server_key: "e274781ab8833745e2e75b46628af289"
server_key_iv: "0683bbabe02c297b2a1d562436633cdd"
server_key_version: "encryptionkey@0"
I20260807 19:47:14.668069 22262 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:47:14.671746 22278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:14.673087 22262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 19:47:14.673283 22262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "309069a3fc254f638268ed213d48cec3"
format_stamp: "Formatted at 2026-08-07 19:47:14 on dist-test-slave-vq2q"
server_key: "e274781ab8833745e2e75b46628af289"
server_key_iv: "0683bbabe02c297b2a1d562436633cdd"
server_key_version: "encryptionkey@0"
I20260807 19:47:14.673487 22262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:14.679533 22262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:14.683955 22262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:14.684253 22262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:14.702971 22262 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38607
I20260807 19:47:14.703065 22334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38607 every 8 connection(s)
I20260807 19:47:14.704385 22262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:14.708185 22335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:14.713666 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 22262
I20260807 19:47:14.713845 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:14.714363 31552 external_mini_cluster.cc:1428] Setting key c85e523092a91d6fc8cd716c48a0d8a3
I20260807 19:47:14.719663 22335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3: Bootstrap starting.
I20260807 19:47:14.722014 22339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:14.716171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46237 (local address 127.0.0.1:38607)
0807 19:47:14.716972 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:14.716987 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:14.717028 (+ 41us) server_negotiation.cc:407] Connection header received
0807 19:47:14.718797 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:14.718821 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:14.719155 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:14.719488 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:14.720528 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:14.720534 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:14.720547 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:14.720560 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:14.720621 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:14.720641 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:14.720807 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:14.720901 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 19:47:14.721061 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":360,"threads_started":1}
I20260807 19:47:14.723061 22335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:14.724077 22335 log.cc:824] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:14.725636 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:14.727444 22335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3: No bootstrap required, opened a new log
I20260807 19:47:14.733110 22339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:14.730853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45563 (local address 127.0.0.1:38607)
0807 19:47:14.731007 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:14.731011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:14.731258 (+ 247us) server_negotiation.cc:407] Connection header received
0807 19:47:14.731440 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:14.731444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:14.731492 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:14.731568 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:14.732476 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:14.732480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:14.732483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:14.732496 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:14.732540 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:14.732555 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:14.732644 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:14.732761 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 19:47:14.732809 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 19:47:14.736855 22335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309069a3fc254f638268ed213d48cec3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } }
I20260807 19:47:14.737268 22335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:14.737363 22335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 309069a3fc254f638268ed213d48cec3, State: Initialized, Role: FOLLOWER
I20260807 19:47:14.737919 22335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309069a3fc254f638268ed213d48cec3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } }
I20260807 19:47:14.738147 22335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:14.738278 22335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:14.738433 22335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:14.740718 22335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309069a3fc254f638268ed213d48cec3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } }
I20260807 19:47:14.741256 22335 leader_election.cc:302] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 309069a3fc254f638268ed213d48cec3; no voters:
I20260807 19:47:14.741829 22335 leader_election.cc:288] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:14.741955 22348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:14.742560 22348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [term 1 LEADER]: Becoming Leader. State: Replica: 309069a3fc254f638268ed213d48cec3, State: Running, Role: LEADER
I20260807 19:47:14.743062 22348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309069a3fc254f638268ed213d48cec3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } }
I20260807 19:47:14.743818 22335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:14.745744 22351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "309069a3fc254f638268ed213d48cec3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309069a3fc254f638268ed213d48cec3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } } }
I20260807 19:47:14.746330 22351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:14.746207 22352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 309069a3fc254f638268ed213d48cec3. Latest consensus state: current_term: 1 leader_uuid: "309069a3fc254f638268ed213d48cec3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309069a3fc254f638268ed213d48cec3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38607 } } }
I20260807 19:47:14.746480 22352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3 [sys.catalog]: This master's current role is: LEADER
W20260807 19:47:14.755964 22367 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:47:14.756085 22367 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:47:14.756213 22358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:14.757929 22358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:14.766527 22358 catalog_manager.cc:1347] Generated new cluster ID: fb6094ec31034323838d63aa49320580
I20260807 19:47:14.766651 22358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:14.784247 22358 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:14.784777 22358 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:14.796921 22358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 309069a3fc254f638268ed213d48cec3: Generated new TSK 0
I20260807 19:47:14.797873 22358 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:14.815495 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 22262
W20260807 19:47:14.816594 22330 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46237
W20260807 19:47:14.817314 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 22262: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:47:14.817468 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 22262
2026-08-07T19:47:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1952 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:14 dist-test-slave-vq2q krb5kdc[22372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:14 dist-test-slave-vq2q krb5kdc[22372](info): set up 2 sockets
Aug 07 19:47:14 dist-test-slave-vq2q krb5kdc[22372](info): commencing operation
krb5kdc: starting...
W20260807 19:47:15.912000 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:16 dist-test-slave-vq2q krb5kdc[22372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:47:16.219764 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40693
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:16.554206 22386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:16.554524 22386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:16.554648 22386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:16.561961 22386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:16.562074 22386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:16.562145 22386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:16.562227 22386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:16.562284 22386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:16.572269 22386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:16.573640 22386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:16.574924 22386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:16.586386 22391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:16.586532 22392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:16.586545 22395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:16.587481 22386 server_base.cc:1034] running on GCE node
I20260807 19:47:16.588208 22386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:16.589622 22386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:16.590859 22386 hybrid_clock.cc:648] HybridClock initialized: now 1786132036590822 us; error 28 us; skew 500 ppm
I20260807 19:47:16.591179 22386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:16 dist-test-slave-vq2q krb5kdc[22372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132036, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:16.594244 22386 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:47:16.595747 22386 webserver.cc:460] Webserver started at http://127.0.0.1:45847/ using document root <none> and password file <none>
I20260807 19:47:16.596408 22386 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:16.596514 22386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:16.596820 22386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:16.599870 22386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "35cb0576595045cf8f757ff8cc75da81"
format_stamp: "Formatted at 2026-08-07 19:47:16 on dist-test-slave-vq2q"
server_key: "433af635efbe4b16e86f9254ab35663e"
server_key_iv: "3e78d1f7f14919863ecb120990e4be8c"
server_key_version: "encryptionkey@0"
I20260807 19:47:16.600631 22386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "35cb0576595045cf8f757ff8cc75da81"
format_stamp: "Formatted at 2026-08-07 19:47:16 on dist-test-slave-vq2q"
server_key: "433af635efbe4b16e86f9254ab35663e"
server_key_iv: "3e78d1f7f14919863ecb120990e4be8c"
server_key_version: "encryptionkey@0"
I20260807 19:47:16.605711 22386 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 19:47:16.609064 22402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:16.610339 22386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:47:16.610500 22386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "35cb0576595045cf8f757ff8cc75da81"
format_stamp: "Formatted at 2026-08-07 19:47:16 on dist-test-slave-vq2q"
server_key: "433af635efbe4b16e86f9254ab35663e"
server_key_iv: "3e78d1f7f14919863ecb120990e4be8c"
server_key_version: "encryptionkey@0"
I20260807 19:47:16.610684 22386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:16.627199 22386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:16.631620 22386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:16.631872 22386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:16.648775 22386 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40693
I20260807 19:47:16.648805 22463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40693 every 8 connection(s)
I20260807 19:47:16.650252 22386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:16.654060 22464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:16.659587 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 22386
I20260807 19:47:16.659783 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:16.660300 31552 external_mini_cluster.cc:1428] Setting key 6910dc1fc594613cc245b87e811f4c14
I20260807 19:47:16.665136 22464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81: Bootstrap starting.
I20260807 19:47:16.668676 22464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:16.669929 22464 log.cc:824] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:16.673332 22464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81: No bootstrap required, opened a new log
Aug 07 19:47:16 dist-test-slave-vq2q krb5kdc[22372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:47:16.682515 22464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35cb0576595045cf8f757ff8cc75da81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40693 } }
I20260807 19:47:16.683018 22464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:16.683187 22464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35cb0576595045cf8f757ff8cc75da81, State: Initialized, Role: FOLLOWER
I20260807 19:47:16.683799 22464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35cb0576595045cf8f757ff8cc75da81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40693 } }
I20260807 19:47:16.684051 22464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:16.684181 22464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:16.684362 22464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:16.684531 22467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:16.662243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.26:45185 (local address 127.0.0.1:40693)
0807 19:47:16.663057 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:16.663071 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:16.663114 (+ 43us) server_negotiation.cc:407] Connection header received
0807 19:47:16.664810 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:16.664839 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:16.665289 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:16.665616 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:16.667363 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:16.668623 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:16.670439 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:16.671464 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:16.671591 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:16.676703 (+ 5112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:16.676711 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:16.676727 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:16.676768 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:16.679907 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:16.680917 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:16.680924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:16.680934 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:16.681114 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:16.681584 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:16.681588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:16.681590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:16.682156 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:16.682399 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:16.682918 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 19:47:16.683296 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":367,"threads_started":1}
I20260807 19:47:16.687479 22464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35cb0576595045cf8f757ff8cc75da81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40693 } }
I20260807 19:47:16.688059 22464 leader_election.cc:302] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35cb0576595045cf8f757ff8cc75da81; no voters:
I20260807 19:47:16.688712 22464 leader_election.cc:288] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:16.688910 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:16.688855 22470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:16.691104 22464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:16.694056 22470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [term 1 LEADER]: Becoming Leader. State: Replica: 35cb0576595045cf8f757ff8cc75da81, State: Running, Role: LEADER
I20260807 19:47:16.694872 22470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35cb0576595045cf8f757ff8cc75da81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40693 } }
I20260807 19:47:16.698102 22471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35cb0576595045cf8f757ff8cc75da81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35cb0576595045cf8f757ff8cc75da81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40693 } } }
I20260807 19:47:16.698606 22471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:16.699180 22472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35cb0576595045cf8f757ff8cc75da81. Latest consensus state: current_term: 1 leader_uuid: "35cb0576595045cf8f757ff8cc75da81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35cb0576595045cf8f757ff8cc75da81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40693 } } }
I20260807 19:47:16.699486 22472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:16.703603 22482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:16.710232 22482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:16.721638 22467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:16.703004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.26:55135 (local address 127.0.0.1:40693)
0807 19:47:16.703190 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:16.703197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:16.703691 (+ 494us) server_negotiation.cc:407] Connection header received
0807 19:47:16.703877 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:16.703883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:16.703985 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:16.704094 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:16.705686 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:16.706984 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:16.708816 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:16.709538 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:16.709669 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:16.713214 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:16.713223 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:16.713228 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:16.713276 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:16.717290 (+ 4014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:16.718346 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:16.718351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:16.718355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:16.718517 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:16.719121 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:16.719126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:16.719129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:16.719622 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:16.719921 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:16.721033 (+ 1112us) server_negotiation.cc:299] Negotiation successful
0807 19:47:16.721267 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 19:47:16.722623 22482 catalog_manager.cc:1347] Generated new cluster ID: 23630e5bbc6c4b57b73ccdd8214a5b52
I20260807 19:47:16.722738 22482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:16.735411 22482 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:16.736730 22482 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:16.745865 22482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35cb0576595045cf8f757ff8cc75da81: Generated new TSK 0
I20260807 19:47:16.747040 22482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:16.757503 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 22386
I20260807 19:47:16.759192 22458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:16.851565 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 22386
W20260807 19:47:16.868454 22370 connection.cc:535] client connection to 127.0.0.1:40693 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40693): Connection reset by peer (error 104)
2026-08-07T19:47:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2033 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:16 dist-test-slave-vq2q krb5kdc[22497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:16 dist-test-slave-vq2q krb5kdc[22497](info): set up 2 sockets
Aug 07 19:47:16 dist-test-slave-vq2q krb5kdc[22497](info): commencing operation
krb5kdc: starting...
W20260807 19:47:17.955884 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:18 dist-test-slave-vq2q krb5kdc[22497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:47:18.292513 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42431
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:18.644987 22514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:18.645386 22514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:18.645538 22514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:18.653030 22514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:18.653159 22514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:18.653249 22514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:18.653348 22514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:18.653409 22514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:18.663741 22514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:18.665128 22514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:18.666821 22514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:18.679157 22520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:18.679160 22519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:18.679157 22522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:18.679630 22514 server_base.cc:1034] running on GCE node
I20260807 19:47:18.680667 22514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:18.682260 22514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:18.683488 22514 hybrid_clock.cc:648] HybridClock initialized: now 1786132038683425 us; error 54 us; skew 500 ppm
I20260807 19:47:18.683866 22514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:18 dist-test-slave-vq2q krb5kdc[22497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132038, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:18.687278 22514 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:47:18.688988 22514 webserver.cc:460] Webserver started at http://127.0.0.1:38715/ using document root <none> and password file <none>
I20260807 19:47:18.689759 22514 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:18.689882 22514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:18.690160 22514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:18.693390 22514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "f0912877ddac45fc83783d8766181e9d"
format_stamp: "Formatted at 2026-08-07 19:47:18 on dist-test-slave-vq2q"
server_key: "b486bfa65c0f970a4736b87318b7f2b2"
server_key_iv: "77306f50903e6a2a2ac9ebcf9ae4062d"
server_key_version: "encryptionkey@0"
I20260807 19:47:18.694205 22514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "f0912877ddac45fc83783d8766181e9d"
format_stamp: "Formatted at 2026-08-07 19:47:18 on dist-test-slave-vq2q"
server_key: "b486bfa65c0f970a4736b87318b7f2b2"
server_key_iv: "77306f50903e6a2a2ac9ebcf9ae4062d"
server_key_version: "encryptionkey@0"
I20260807 19:47:18.700773 22514 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 19:47:18.705117 22529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:18.707029 22514 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 19:47:18.707273 22514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "f0912877ddac45fc83783d8766181e9d"
format_stamp: "Formatted at 2026-08-07 19:47:18 on dist-test-slave-vq2q"
server_key: "b486bfa65c0f970a4736b87318b7f2b2"
server_key_iv: "77306f50903e6a2a2ac9ebcf9ae4062d"
server_key_version: "encryptionkey@0"
I20260807 19:47:18.707520 22514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:18.731956 22514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:18.736531 22514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:18.736789 22514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:18.754575 22514 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42431
I20260807 19:47:18.754599 22587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42431 every 8 connection(s)
I20260807 19:47:18.755972 22514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:18.759930 22588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:18.764066 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 22514
I20260807 19:47:18.764214 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:18.764642 31552 external_mini_cluster.cc:1428] Setting key 9eac958c7625bd206d1c9259329dd898
I20260807 19:47:18.771267 22588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d: Bootstrap starting.
I20260807 19:47:18.775081 22588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:18.776368 22588 log.cc:824] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:18.779549 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:18.766507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.26:37783 (local address 127.0.0.1:42431)
0807 19:47:18.767300 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:18.767312 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:18.767354 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:47:18.769753 (+ 2399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:18.769778 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:18.770192 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:18.770533 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:18.771244 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:18.772643 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:18.774687 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:18.775454 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:18.775607 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:18.776881 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:18.776891 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:18.776908 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:18.777736 (+ 828us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:18.777835 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:18.777871 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:18.778007 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:18.778390 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 19:47:18.778583 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":345,"threads_started":1}
I20260807 19:47:18.779716 22588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d: No bootstrap required, opened a new log
I20260807 19:47:18.783494 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:18.789333 22588 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0912877ddac45fc83783d8766181e9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42431 } }
I20260807 19:47:18.789834 22588 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:18.789958 22588 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0912877ddac45fc83783d8766181e9d, State: Initialized, Role: FOLLOWER
I20260807 19:47:18.790650 22588 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0912877ddac45fc83783d8766181e9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42431 } }
I20260807 19:47:18.790961 22588 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:18.791119 22588 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:18.791299 22588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:18.794088 22588 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0912877ddac45fc83783d8766181e9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42431 } }
I20260807 19:47:18.794602 22588 leader_election.cc:302] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0912877ddac45fc83783d8766181e9d; no voters:
I20260807 19:47:18.795279 22588 leader_election.cc:288] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:18.795464 22598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:18.795970 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:18.788847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.26:36035 (local address 127.0.0.1:42431)
0807 19:47:18.789046 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:18.789052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:18.789568 (+ 516us) server_negotiation.cc:407] Connection header received
0807 19:47:18.789745 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:18.789750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:18.789843 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:18.789961 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:18.790508 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:18.791630 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:18.792977 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:18.793715 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:18.793850 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:18.794978 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:18.794984 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:18.794988 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:18.795012 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:18.795072 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:18.795096 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:18.795214 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:18.795547 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 19:47:18.795606 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 19:47:18.796478 22598 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [term 1 LEADER]: Becoming Leader. State: Replica: f0912877ddac45fc83783d8766181e9d, State: Running, Role: LEADER
I20260807 19:47:18.797140 22598 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0912877ddac45fc83783d8766181e9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42431 } }
I20260807 19:47:18.798810 22588 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:18.799959 22599 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0912877ddac45fc83783d8766181e9d. Latest consensus state: current_term: 1 leader_uuid: "f0912877ddac45fc83783d8766181e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0912877ddac45fc83783d8766181e9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42431 } } }
I20260807 19:47:18.800391 22599 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:18.800699 22600 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0912877ddac45fc83783d8766181e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0912877ddac45fc83783d8766181e9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42431 } } }
I20260807 19:47:18.801005 22600 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:18.806025 22603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:18.811523 22603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:18.824769 22603 catalog_manager.cc:1347] Generated new cluster ID: d9d5f06f3ee64adaa25bae113200031b
I20260807 19:47:18.824901 22603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:18.843235 22603 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:18.844578 22603 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:18.855150 22603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0912877ddac45fc83783d8766181e9d: Generated new TSK 0
I20260807 19:47:18.856139 22603 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:18.864375 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 22514
W20260807 19:47:18.865713 22583 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.26:37783
W20260807 19:47:18.866499 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 22514: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:47:18.866662 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 22514
W20260807 19:47:18.885485 22495 connection.cc:535] client connection to 127.0.0.1:42431 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42431): Connection reset by peer (error 104)
2026-08-07T19:47:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2017 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:18 dist-test-slave-vq2q krb5kdc[22621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:18 dist-test-slave-vq2q krb5kdc[22621](info): set up 2 sockets
Aug 07 19:47:18 dist-test-slave-vq2q krb5kdc[22621](info): commencing operation
krb5kdc: starting...
W20260807 19:47:19.971437 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:20 dist-test-slave-vq2q krb5kdc[22621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:47:20.281335 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40167
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:20.620723 22634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:20.621002 22634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:20.621095 22634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:20.628221 22634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:20.628328 22634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:20.628378 22634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:20.628443 22634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:20.628485 22634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:20.638432 22634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:20.639689 22634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:20.640964 22634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:20.652501 22639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:20.652438 22643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:20.652475 22640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:20.653369 22634 server_base.cc:1034] running on GCE node
I20260807 19:47:20.654124 22634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:20.655494 22634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:20.656693 22634 hybrid_clock.cc:648] HybridClock initialized: now 1786132040656644 us; error 45 us; skew 500 ppm
I20260807 19:47:20.657022 22634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:20 dist-test-slave-vq2q krb5kdc[22621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132040, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:20.662791 22634 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:47:20.664256 22634 webserver.cc:460] Webserver started at http://127.0.0.1:41127/ using document root <none> and password file <none>
I20260807 19:47:20.664906 22634 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:20.665022 22634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:20.665359 22634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:20.668320 22634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "814958b081dd471bae1733434a491fb0"
format_stamp: "Formatted at 2026-08-07 19:47:20 on dist-test-slave-vq2q"
server_key: "4cd329940bf5a621706981f103a31382"
server_key_iv: "4457b44f740fb58bba8ef164c6ab8ab4"
server_key_version: "encryptionkey@0"
I20260807 19:47:20.669070 22634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "814958b081dd471bae1733434a491fb0"
format_stamp: "Formatted at 2026-08-07 19:47:20 on dist-test-slave-vq2q"
server_key: "4cd329940bf5a621706981f103a31382"
server_key_iv: "4457b44f740fb58bba8ef164c6ab8ab4"
server_key_version: "encryptionkey@0"
I20260807 19:47:20.673987 22634 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 19:47:20.677268 22651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:20.678508 22634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 19:47:20.678661 22634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "814958b081dd471bae1733434a491fb0"
format_stamp: "Formatted at 2026-08-07 19:47:20 on dist-test-slave-vq2q"
server_key: "4cd329940bf5a621706981f103a31382"
server_key_iv: "4457b44f740fb58bba8ef164c6ab8ab4"
server_key_version: "encryptionkey@0"
I20260807 19:47:20.678851 22634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:20.684584 22634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:20.688820 22634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:20.689110 22634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:20.707489 22634 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40167
I20260807 19:47:20.707532 22707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40167 every 8 connection(s)
I20260807 19:47:20.708875 22634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:20.710870 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 22634
I20260807 19:47:20.711066 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:20.711632 31552 external_mini_cluster.cc:1428] Setting key 66f903be21df8c0b5a43abdb298939a8
I20260807 19:47:20.713406 22708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:20.719314 22714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:20.713467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.26:59605 (local address 127.0.0.1:40167)
0807 19:47:20.714253 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:20.714264 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:20.714304 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:47:20.715922 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:20.715943 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:20.716294 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:20.716625 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:20.717710 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:20.717716 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:20.717729 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:20.717744 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:20.717851 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:20.717876 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:20.718061 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:20.718169 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 19:47:20.718358 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":339,"threads_started":1}
I20260807 19:47:20.722841 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:20.726439 22708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0: Bootstrap starting.
I20260807 19:47:20.730583 22708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:20.730612 22714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:20.728274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.26:34287 (local address 127.0.0.1:40167)
0807 19:47:20.728437 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:20.728442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:20.728769 (+ 327us) server_negotiation.cc:407] Connection header received
0807 19:47:20.728954 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:20.728957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:20.729006 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:20.729083 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:20.730016 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:20.730021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:20.730023 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:20.730038 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:20.730078 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:20.730092 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:20.730173 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:20.730295 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 19:47:20.730346 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 19:47:20.731710 22708 log.cc:824] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:20.734434 22708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0: No bootstrap required, opened a new log
I20260807 19:47:20.741657 22708 raft_consensus.cc:348] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814958b081dd471bae1733434a491fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40167 } }
I20260807 19:47:20.742054 22708 raft_consensus.cc:374] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:20.742152 22708 raft_consensus.cc:729] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 814958b081dd471bae1733434a491fb0, State: Initialized, Role: FOLLOWER
I20260807 19:47:20.742697 22708 consensus_queue.cc:260] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814958b081dd471bae1733434a491fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40167 } }
I20260807 19:47:20.742923 22708 raft_consensus.cc:388] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:20.743042 22708 raft_consensus.cc:482] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:20.743191 22708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:20.745394 22708 raft_consensus.cc:504] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814958b081dd471bae1733434a491fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40167 } }
I20260807 19:47:20.745890 22708 leader_election.cc:302] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 814958b081dd471bae1733434a491fb0; no voters:
I20260807 19:47:20.746492 22708 leader_election.cc:288] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:20.746613 22722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:20.747236 22722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [term 1 LEADER]: Becoming Leader. State: Replica: 814958b081dd471bae1733434a491fb0, State: Running, Role: LEADER
I20260807 19:47:20.747795 22722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814958b081dd471bae1733434a491fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40167 } }
I20260807 19:47:20.748788 22708 sys_catalog.cc:564] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:20.750751 22723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "814958b081dd471bae1733434a491fb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814958b081dd471bae1733434a491fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40167 } } }
I20260807 19:47:20.750875 22725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 814958b081dd471bae1733434a491fb0. Latest consensus state: current_term: 1 leader_uuid: "814958b081dd471bae1733434a491fb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "814958b081dd471bae1733434a491fb0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40167 } } }
I20260807 19:47:20.751152 22723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:20.751216 22725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:20.753062 22731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:20.757871 22731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:20.766222 22731 catalog_manager.cc:1347] Generated new cluster ID: 99c87524aea04829a1ffc16b487e4475
I20260807 19:47:20.766352 22731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:20.780732 22731 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:20.781308 22731 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:20.790048 22731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 814958b081dd471bae1733434a491fb0: Generated new TSK 0
I20260807 19:47:20.791146 22731 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:20.798213 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 22634
W20260807 19:47:20.799520 22703 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.26:59605
W20260807 19:47:20.800205 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 22634: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:47:20.800371 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 22634
2026-08-07T19:47:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1932 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:20 dist-test-slave-vq2q krb5kdc[22746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:20 dist-test-slave-vq2q krb5kdc[22746](info): set up 2 sockets
Aug 07 19:47:20 dist-test-slave-vq2q krb5kdc[22746](info): commencing operation
krb5kdc: starting...
W20260807 19:47:21.900115 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.005s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:22 dist-test-slave-vq2q krb5kdc[22746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:47:22.212517 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38097
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38097
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:22.559520 22760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:22.559835 22760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:22.559976 22760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:22.567663 22760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:22.567795 22760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:22.567880 22760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:22.567963 22760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:22.568018 22760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:22.578437 22760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38097
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:22.579733 22760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:22.581027 22760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:22.592345 22765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:22.592314 22767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:22.592267 22770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:22.593617 22760 server_base.cc:1034] running on GCE node
I20260807 19:47:22.594436 22760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:22.595880 22760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:22.597080 22760 hybrid_clock.cc:648] HybridClock initialized: now 1786132042597033 us; error 41 us; skew 500 ppm
I20260807 19:47:22.597455 22760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:22 dist-test-slave-vq2q krb5kdc[22746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132042, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:22.600522 22760 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:47:22.602108 22760 webserver.cc:460] Webserver started at http://127.0.0.1:45285/ using document root <none> and password file <none>
I20260807 19:47:22.602759 22760 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:22.602878 22760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:22.603169 22760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:22.606220 22760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "775280a1585546d98b37b5e04418f895"
format_stamp: "Formatted at 2026-08-07 19:47:22 on dist-test-slave-vq2q"
server_key: "fabe0044213a7d9a4f0d368ace2302e0"
server_key_iv: "5abb9167c745795d59730ec23a95aff9"
server_key_version: "encryptionkey@0"
I20260807 19:47:22.606982 22760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "775280a1585546d98b37b5e04418f895"
format_stamp: "Formatted at 2026-08-07 19:47:22 on dist-test-slave-vq2q"
server_key: "fabe0044213a7d9a4f0d368ace2302e0"
server_key_iv: "5abb9167c745795d59730ec23a95aff9"
server_key_version: "encryptionkey@0"
I20260807 19:47:22.612361 22760 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:47:22.615890 22778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:22.617244 22760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:47:22.617410 22760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "775280a1585546d98b37b5e04418f895"
format_stamp: "Formatted at 2026-08-07 19:47:22 on dist-test-slave-vq2q"
server_key: "fabe0044213a7d9a4f0d368ace2302e0"
server_key_iv: "5abb9167c745795d59730ec23a95aff9"
server_key_version: "encryptionkey@0"
I20260807 19:47:22.617592 22760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:22.631935 22760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:22.636415 22760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:22.636683 22760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:22.655385 22760 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38097
I20260807 19:47:22.655454 22838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38097 every 8 connection(s)
I20260807 19:47:22.656785 22760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:22.660645 22839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:22.663519 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 22760
I20260807 19:47:22.663703 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:22.664275 31552 external_mini_cluster.cc:1428] Setting key d0942a6e0b1057b065271ca0e40928ca
I20260807 19:47:22.672655 22839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895: Bootstrap starting.
I20260807 19:47:22.675982 22839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:22.677037 22839 log.cc:824] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895: Log is configured to *not* fsync() on all Append() calls
Aug 07 19:47:22 dist-test-slave-vq2q krb5kdc[22746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 19:47:22.680001 22839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895: No bootstrap required, opened a new log
I20260807 19:47:22.687005 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.666218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.26:48641 (local address 127.0.0.1:38097)
0807 19:47:22.667088 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:22.667101 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:22.667143 (+ 42us) server_negotiation.cc:407] Connection header received
0807 19:47:22.668827 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:22.668849 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:22.669307 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:22.669672 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:22.671181 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.672525 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.673867 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.674580 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.674750 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.679075 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:22.679084 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:22.679099 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:22.679134 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:22.682332 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.683218 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.683228 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.683235 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.683451 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.683936 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.683939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.683941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.684577 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:22.684931 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:22.685511 (+ 580us) server_negotiation.cc:299] Negotiation successful
0807 19:47:22.685878 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":369,"threads_started":1}
I20260807 19:47:22.688544 22839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775280a1585546d98b37b5e04418f895" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38097 } }
I20260807 19:47:22.689002 22839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:22.689113 22839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 775280a1585546d98b37b5e04418f895, State: Initialized, Role: FOLLOWER
I20260807 19:47:22.689805 22839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775280a1585546d98b37b5e04418f895" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38097 } }
I20260807 19:47:22.690116 22839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:22.690269 22839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:22.690448 22839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:22.691705 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:22.692978 22839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775280a1585546d98b37b5e04418f895" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38097 } }
I20260807 19:47:22.693507 22839 leader_election.cc:302] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 775280a1585546d98b37b5e04418f895; no voters:
I20260807 19:47:22.694120 22839 leader_election.cc:288] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:22.694314 22844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:22.696226 22839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:22.703976 22844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [term 1 LEADER]: Becoming Leader. State: Replica: 775280a1585546d98b37b5e04418f895, State: Running, Role: LEADER
I20260807 19:47:22.704732 22844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775280a1585546d98b37b5e04418f895" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38097 } }
I20260807 19:47:22.707775 22846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "775280a1585546d98b37b5e04418f895" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775280a1585546d98b37b5e04418f895" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38097 } } }
I20260807 19:47:22.708220 22846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:22.709658 22848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 775280a1585546d98b37b5e04418f895. Latest consensus state: current_term: 1 leader_uuid: "775280a1585546d98b37b5e04418f895" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "775280a1585546d98b37b5e04418f895" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38097 } } }
I20260807 19:47:22.710438 22848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895 [sys.catalog]: This master's current role is: LEADER
W20260807 19:47:22.715157 22867 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:47:22.715298 22867 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:47:22.715449 22866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:22.717764 22866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:22.722419 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:22.706401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.26:45311 (local address 127.0.0.1:38097)
0807 19:47:22.706633 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:22.706641 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:22.707176 (+ 535us) server_negotiation.cc:407] Connection header received
0807 19:47:22.707486 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:22.707491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:22.707572 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:22.707658 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:22.709411 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.710465 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.711861 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:22.712360 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:22.712481 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:22.714953 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:22.714959 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:22.714962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:22.714993 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:22.718857 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.719865 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.719870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.719873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.720020 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:22.720529 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:22.720578 (+ 49us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:22.720583 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:22.721059 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:22.721371 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:22.721884 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 19:47:22.722059 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:47:22.727232 22866 catalog_manager.cc:1347] Generated new cluster ID: 704345f5908a48a59cd49df8238c3219
I20260807 19:47:22.727393 22866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:22.747905 22866 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:22.749888 22866 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:22.758908 22866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 775280a1585546d98b37b5e04418f895: Generated new TSK 0
I20260807 19:47:22.760025 22866 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:22.774876 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 22760
I20260807 19:47:22.776644 22832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:22.870447 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 22760
W20260807 19:47:22.888471 22744 connection.cc:535] client connection to 127.0.0.1:38097 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38097): Connection reset by peer (error 104)
2026-08-07T19:47:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2069 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:22 dist-test-slave-vq2q krb5kdc[22877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:22 dist-test-slave-vq2q krb5kdc[22877](info): set up 2 sockets
Aug 07 19:47:22 dist-test-slave-vq2q krb5kdc[22877](info): commencing operation
krb5kdc: starting...
W20260807 19:47:23.972132 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:24 dist-test-slave-vq2q krb5kdc[22877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132044, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:47:24.284453 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42505
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:24.622120 22890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:24.622417 22890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:24.622550 22890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:24.629957 22890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:24.630081 22890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:24.630141 22890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:24.630228 22890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:24.630285 22890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:24.640715 22890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42505
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:24.642292 22890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:24.643832 22890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:24.659333 22896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:24.659407 22895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:24.660014 22898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:24.660449 22890 server_base.cc:1034] running on GCE node
I20260807 19:47:24.661363 22890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:24.663028 22890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:24.664250 22890 hybrid_clock.cc:648] HybridClock initialized: now 1786132044664196 us; error 48 us; skew 500 ppm
I20260807 19:47:24.664639 22890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:24 dist-test-slave-vq2q krb5kdc[22877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132044, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:24.668047 22890 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:47:24.669755 22890 webserver.cc:460] Webserver started at http://127.0.0.1:45477/ using document root <none> and password file <none>
I20260807 19:47:24.670524 22890 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:24.670655 22890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:24.670960 22890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:24.674263 22890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "db7d7233aab04818a57018e68cd8e8ba"
format_stamp: "Formatted at 2026-08-07 19:47:24 on dist-test-slave-vq2q"
server_key: "6e82126403c7261b5a9e8604ec2aa1a9"
server_key_iv: "99bc58ef1f0d66798e3d5b7d2f4b9cae"
server_key_version: "encryptionkey@0"
I20260807 19:47:24.675081 22890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "db7d7233aab04818a57018e68cd8e8ba"
format_stamp: "Formatted at 2026-08-07 19:47:24 on dist-test-slave-vq2q"
server_key: "6e82126403c7261b5a9e8604ec2aa1a9"
server_key_iv: "99bc58ef1f0d66798e3d5b7d2f4b9cae"
server_key_version: "encryptionkey@0"
I20260807 19:47:24.681274 22890 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 19:47:24.685822 22906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:24.688083 22890 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260807 19:47:24.688321 22890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "db7d7233aab04818a57018e68cd8e8ba"
format_stamp: "Formatted at 2026-08-07 19:47:24 on dist-test-slave-vq2q"
server_key: "6e82126403c7261b5a9e8604ec2aa1a9"
server_key_iv: "99bc58ef1f0d66798e3d5b7d2f4b9cae"
server_key_version: "encryptionkey@0"
I20260807 19:47:24.688572 22890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:24.711489 22890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:24.716632 22890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:24.716950 22890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:24.739356 22890 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42505
I20260807 19:47:24.739391 22963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42505 every 8 connection(s)
I20260807 19:47:24.740872 22890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:24.744724 22964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:24.745878 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 22890
I20260807 19:47:24.746057 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:24.746505 31552 external_mini_cluster.cc:1428] Setting key 44a8384e29ed0c3170b4ac2ec6008b83
I20260807 19:47:24.759332 22964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba: Bootstrap starting.
I20260807 19:47:24.761626 22967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:24.748291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.26:49333 (local address 127.0.0.1:42505)
0807 19:47:24.749055 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:24.749071 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:24.749116 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:47:24.751461 (+ 2345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:24.751521 (+ 60us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:24.752355 (+ 834us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:24.752826 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:24.753481 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:24.754828 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:24.756800 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:24.757591 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:24.757746 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:24.759067 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:24.759075 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:24.759090 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:24.759669 (+ 579us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:24.759818 (+ 149us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:24.759848 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:24.759970 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:24.760242 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 19:47:24.760424 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":328,"threads_started":1}
I20260807 19:47:24.763082 22964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:24.764319 22964 log.cc:824] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:24.767103 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:24.768157 22964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba: No bootstrap required, opened a new log
I20260807 19:47:24.778227 22964 raft_consensus.cc:348] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d7233aab04818a57018e68cd8e8ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42505 } }
I20260807 19:47:24.778789 22964 raft_consensus.cc:374] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:24.778915 22964 raft_consensus.cc:729] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db7d7233aab04818a57018e68cd8e8ba, State: Initialized, Role: FOLLOWER
I20260807 19:47:24.779316 22967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:24.772318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.26:43747 (local address 127.0.0.1:42505)
0807 19:47:24.772548 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:24.772555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:24.772972 (+ 417us) server_negotiation.cc:407] Connection header received
0807 19:47:24.773123 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:24.773130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:24.773262 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:24.773382 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:24.773953 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:24.775129 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:24.776442 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:24.777145 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:24.777300 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:24.778393 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:24.778399 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:24.778403 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:24.778427 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:24.778494 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:24.778517 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:24.778644 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:24.778926 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 19:47:24.778987 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:47:24.779640 22964 consensus_queue.cc:260] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d7233aab04818a57018e68cd8e8ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42505 } }
I20260807 19:47:24.779901 22964 raft_consensus.cc:388] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:24.780045 22964 raft_consensus.cc:482] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:24.780212 22964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:24.782519 22964 raft_consensus.cc:504] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d7233aab04818a57018e68cd8e8ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42505 } }
I20260807 19:47:24.783031 22964 leader_election.cc:302] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db7d7233aab04818a57018e68cd8e8ba; no voters:
I20260807 19:47:24.783771 22964 leader_election.cc:288] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:24.784008 22974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:24.784669 22974 raft_consensus.cc:686] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [term 1 LEADER]: Becoming Leader. State: Replica: db7d7233aab04818a57018e68cd8e8ba, State: Running, Role: LEADER
I20260807 19:47:24.785298 22974 consensus_queue.cc:237] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d7233aab04818a57018e68cd8e8ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42505 } }
I20260807 19:47:24.785985 22964 sys_catalog.cc:564] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:24.787994 22976 sys_catalog.cc:455] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader db7d7233aab04818a57018e68cd8e8ba. Latest consensus state: current_term: 1 leader_uuid: "db7d7233aab04818a57018e68cd8e8ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d7233aab04818a57018e68cd8e8ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42505 } } }
I20260807 19:47:24.788421 22976 sys_catalog.cc:458] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:24.789359 22984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:24.789276 22975 sys_catalog.cc:455] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db7d7233aab04818a57018e68cd8e8ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d7233aab04818a57018e68cd8e8ba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42505 } } }
I20260807 19:47:24.790154 22975 sys_catalog.cc:458] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:24.795101 22984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:24.805325 22984 catalog_manager.cc:1347] Generated new cluster ID: c06d49186c8840d2849d2a73875eb559
I20260807 19:47:24.805442 22984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:24.832731 22984 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:24.834717 22984 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:24.846387 22984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db7d7233aab04818a57018e68cd8e8ba: Generated new TSK 0
I20260807 19:47:24.847474 22984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:24.867175 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 22890
W20260807 19:47:24.869063 22958 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.26:49333
W20260807 19:47:24.869794 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 22890: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:47:24.869928 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 22890
W20260807 19:47:24.887446 22875 connection.cc:535] client connection to 127.0.0.1:42505 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42505): Connection reset by peer (error 104)
2026-08-07T19:47:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2003 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:24 dist-test-slave-vq2q krb5kdc[22996](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:24 dist-test-slave-vq2q krb5kdc[22996](info): set up 2 sockets
Aug 07 19:47:24 dist-test-slave-vq2q krb5kdc[22996](info): commencing operation
krb5kdc: starting...
W20260807 19:47:25.976423 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:26 dist-test-slave-vq2q krb5kdc[22996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 19:47:26.282653 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33999
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:26.620987 23009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:26.621299 23009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:26.621443 23009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:26.628779 23009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:26.628904 23009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:26.628963 23009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:26.629045 23009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:26.629106 23009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:26.639247 23009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33999
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:26.640547 23009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:26.641904 23009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:26.653168 23019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:26.653285 23016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:26.653256 23017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:26.654393 23009 server_base.cc:1034] running on GCE node
I20260807 19:47:26.655102 23009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:26.656514 23009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:26.657718 23009 hybrid_clock.cc:648] HybridClock initialized: now 1786132046657667 us; error 44 us; skew 500 ppm
I20260807 19:47:26.658043 23009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:26 dist-test-slave-vq2q krb5kdc[22996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132046, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:26.665241 23009 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 19:47:26.666704 23009 webserver.cc:460] Webserver started at http://127.0.0.1:32771/ using document root <none> and password file <none>
I20260807 19:47:26.667378 23009 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:26.667481 23009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:26.667738 23009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:26.670715 23009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "4b68b04b43654c03be9eada02a89b27e"
format_stamp: "Formatted at 2026-08-07 19:47:26 on dist-test-slave-vq2q"
server_key: "b058ba51642aef54156a5856972c8a68"
server_key_iv: "1b83f8a829978e5686cf19beafb44912"
server_key_version: "encryptionkey@0"
I20260807 19:47:26.671419 23009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "4b68b04b43654c03be9eada02a89b27e"
format_stamp: "Formatted at 2026-08-07 19:47:26 on dist-test-slave-vq2q"
server_key: "b058ba51642aef54156a5856972c8a68"
server_key_iv: "1b83f8a829978e5686cf19beafb44912"
server_key_version: "encryptionkey@0"
I20260807 19:47:26.676643 23009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:47:26.680256 23027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:26.681726 23009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:47:26.681903 23009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "4b68b04b43654c03be9eada02a89b27e"
format_stamp: "Formatted at 2026-08-07 19:47:26 on dist-test-slave-vq2q"
server_key: "b058ba51642aef54156a5856972c8a68"
server_key_iv: "1b83f8a829978e5686cf19beafb44912"
server_key_version: "encryptionkey@0"
I20260807 19:47:26.682092 23009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:26.688092 23009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:26.692644 23009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:26.692968 23009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:26.711545 23009 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33999
I20260807 19:47:26.711588 23083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33999 every 8 connection(s)
I20260807 19:47:26.713105 23009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:26.716766 23084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:26.723007 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 23009
I20260807 19:47:26.723163 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:26.723604 31552 external_mini_cluster.cc:1428] Setting key 9a72907b4e00c57e3f40727cbd06a042
I20260807 19:47:26.728181 23084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e: Bootstrap starting.
I20260807 19:47:26.732159 23084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:26.732483 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:26.725545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.26:51523 (local address 127.0.0.1:33999)
0807 19:47:26.726392 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:26.726407 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:26.726457 (+ 50us) server_negotiation.cc:407] Connection header received
0807 19:47:26.728746 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:26.728775 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:26.729222 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:26.729644 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:26.730646 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:26.730653 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:26.730669 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:26.730687 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:26.730755 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:26.730785 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:26.730980 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:26.731100 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 19:47:26.731289 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":331,"threads_started":1}
I20260807 19:47:26.733419 23084 log.cc:824] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:26.736292 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:26.736397 23084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e: No bootstrap required, opened a new log
I20260807 19:47:26.742844 23087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:26.740595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.26:44101 (local address 127.0.0.1:33999)
0807 19:47:26.740790 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:26.740799 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:26.741005 (+ 206us) server_negotiation.cc:407] Connection header received
0807 19:47:26.741152 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:26.741158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:26.741246 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:26.741352 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:26.742227 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:26.742233 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:26.742237 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:26.742255 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:26.742299 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:26.742317 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:26.742441 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:26.742480 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 19:47:26.742536 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:47:26.744827 23084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b68b04b43654c03be9eada02a89b27e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33999 } }
I20260807 19:47:26.745239 23084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:26.745323 23084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b68b04b43654c03be9eada02a89b27e, State: Initialized, Role: FOLLOWER
I20260807 19:47:26.745920 23084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b68b04b43654c03be9eada02a89b27e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33999 } }
I20260807 19:47:26.746155 23084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:26.746253 23084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:26.746383 23084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:26.748838 23084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b68b04b43654c03be9eada02a89b27e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33999 } }
I20260807 19:47:26.749429 23084 leader_election.cc:302] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b68b04b43654c03be9eada02a89b27e; no voters:
I20260807 19:47:26.750048 23084 leader_election.cc:288] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:26.750139 23096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:26.751274 23096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [term 1 LEADER]: Becoming Leader. State: Replica: 4b68b04b43654c03be9eada02a89b27e, State: Running, Role: LEADER
I20260807 19:47:26.751799 23096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b68b04b43654c03be9eada02a89b27e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33999 } }
I20260807 19:47:26.752156 23084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:26.754684 23097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b68b04b43654c03be9eada02a89b27e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b68b04b43654c03be9eada02a89b27e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33999 } } }
I20260807 19:47:26.754673 23098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b68b04b43654c03be9eada02a89b27e. Latest consensus state: current_term: 1 leader_uuid: "4b68b04b43654c03be9eada02a89b27e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b68b04b43654c03be9eada02a89b27e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33999 } } }
I20260807 19:47:26.755142 23097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:26.755175 23098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:26.756054 23104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:26.762904 23104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:26.774303 23104 catalog_manager.cc:1347] Generated new cluster ID: 2e5f3fb8d28a43ba9815cfcf611a0d82
I20260807 19:47:26.774436 23104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:26.793514 23104 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:26.793978 23104 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:26.800400 23104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b68b04b43654c03be9eada02a89b27e: Generated new TSK 0
I20260807 19:47:26.801326 23104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:26.812613 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 23009
W20260807 19:47:26.813776 23079 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.26:51523
W20260807 19:47:26.814543 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 23009: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:47:26.814713 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 23009
2026-08-07T19:47:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1941 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:26 dist-test-slave-vq2q krb5kdc[23121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:26 dist-test-slave-vq2q krb5kdc[23121](info): set up 2 sockets
Aug 07 19:47:26 dist-test-slave-vq2q krb5kdc[23121](info): commencing operation
krb5kdc: starting...
W20260807 19:47:27.919013 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.004s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:28 dist-test-slave-vq2q krb5kdc[23121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:47:28.224766 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:43725
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:43725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:28.568564 23135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:28.568876 23135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:28.568970 23135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:28.576314 23135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:28.576444 23135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:28.576493 23135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:28.576552 23135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:28.576594 23135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:28.588867 23135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:43725
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:43725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:28.590257 23135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:28.591579 23135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:28.603588 23143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:28.603645 23140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:28.603547 23141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:28.604297 23135 server_base.cc:1034] running on GCE node
I20260807 19:47:28.605257 23135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:28.607118 23135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:28.608426 23135 hybrid_clock.cc:648] HybridClock initialized: now 1786132048608377 us; error 42 us; skew 500 ppm
I20260807 19:47:28.608819 23135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:28 dist-test-slave-vq2q krb5kdc[23121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132048, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:28.612035 23135 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:47:28.613639 23135 webserver.cc:460] Webserver started at http://127.30.208.62:33185/ using document root <none> and password file <none>
I20260807 19:47:28.614383 23135 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:28.614514 23135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:28.614823 23135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:28.618613 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "1b8438ded618475e87b29ae551405fda"
format_stamp: "Formatted at 2026-08-07 19:47:28 on dist-test-slave-vq2q"
server_key: "7384801d0fc3bf43272eb662998218b8"
server_key_iv: "ad706f3ab58f195e2573993abd117a83"
server_key_version: "encryptionkey@0"
I20260807 19:47:28.619489 23135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "1b8438ded618475e87b29ae551405fda"
format_stamp: "Formatted at 2026-08-07 19:47:28 on dist-test-slave-vq2q"
server_key: "7384801d0fc3bf43272eb662998218b8"
server_key_iv: "ad706f3ab58f195e2573993abd117a83"
server_key_version: "encryptionkey@0"
I20260807 19:47:28.625022 23135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:47:28.628767 23150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:28.630188 23135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:47:28.630362 23135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "1b8438ded618475e87b29ae551405fda"
format_stamp: "Formatted at 2026-08-07 19:47:28 on dist-test-slave-vq2q"
server_key: "7384801d0fc3bf43272eb662998218b8"
server_key_iv: "ad706f3ab58f195e2573993abd117a83"
server_key_version: "encryptionkey@0"
I20260807 19:47:28.630553 23135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:28.650287 23135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:28.654565 23135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:28.654841 23135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:28.671885 23135 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:43725
I20260807 19:47:28.671913 23210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:43725 every 8 connection(s)
I20260807 19:47:28.673224 23135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:28.675755 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 23135
I20260807 19:47:28.675923 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:28.676337 31552 external_mini_cluster.cc:1428] Setting key 59aeaa3725e995690d049c48b3a83292
I20260807 19:47:28.676941 23212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:47:28 dist-test-slave-vq2q krb5kdc[23121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:47:28.689352 23212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda: Bootstrap starting.
I20260807 19:47:28.692586 23212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:28.693665 23212 log.cc:824] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:28.696322 23212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda: No bootstrap required, opened a new log
I20260807 19:47:28.697589 23215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:28.678075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44219 (local address 127.30.208.62:43725)
0807 19:47:28.678816 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:28.678831 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:28.678876 (+ 45us) server_negotiation.cc:407] Connection header received
0807 19:47:28.680390 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:28.680411 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:28.680792 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:28.681146 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:28.682583 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:28.683940 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:28.685200 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:28.685896 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:28.686079 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:28.690484 (+ 4405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:28.690492 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:28.690505 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:28.690541 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:28.693524 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:28.694313 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:28.694320 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:28.694329 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:28.694512 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:28.694955 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:28.694959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:28.694961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:28.695453 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:28.695758 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:28.696224 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 19:47:28.696561 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":336,"threads_started":1}
I20260807 19:47:28.701453 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:28.704716 23212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b8438ded618475e87b29ae551405fda" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 43725 } }
I20260807 19:47:28.705097 23212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:28.705220 23212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b8438ded618475e87b29ae551405fda, State: Initialized, Role: FOLLOWER
I20260807 19:47:28.706073 23212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b8438ded618475e87b29ae551405fda" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 43725 } }
I20260807 19:47:28.706333 23212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:28.706478 23212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:28.706658 23212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:28.708850 23212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b8438ded618475e87b29ae551405fda" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 43725 } }
I20260807 19:47:28.709405 23212 leader_election.cc:302] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b8438ded618475e87b29ae551405fda; no voters:
I20260807 19:47:28.710043 23212 leader_election.cc:288] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:28.710147 23222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:28.712641 23212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:28.713474 23222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [term 1 LEADER]: Becoming Leader. State: Replica: 1b8438ded618475e87b29ae551405fda, State: Running, Role: LEADER
I20260807 19:47:28.714172 23222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b8438ded618475e87b29ae551405fda" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 43725 } }
I20260807 19:47:28.717394 23224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b8438ded618475e87b29ae551405fda. Latest consensus state: current_term: 1 leader_uuid: "1b8438ded618475e87b29ae551405fda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b8438ded618475e87b29ae551405fda" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 43725 } } }
I20260807 19:47:28.717620 23223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b8438ded618475e87b29ae551405fda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b8438ded618475e87b29ae551405fda" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 43725 } } }
I20260807 19:47:28.717865 23224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:28.717916 23223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:28.720276 23232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:28.724941 23215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:28.705740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56603 (local address 127.30.208.62:43725)
0807 19:47:28.705926 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:28.705932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:28.706270 (+ 338us) server_negotiation.cc:407] Connection header received
0807 19:47:28.706442 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:28.706447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:28.706521 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:28.706602 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:28.708021 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:28.708777 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:28.710197 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:28.710931 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:28.711658 (+ 727us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:28.713826 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:28.713831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:28.713835 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:28.713868 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:28.720346 (+ 6478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:28.721171 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:28.721205 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:28.721208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:28.721430 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:28.722147 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:28.722152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:28.722155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:28.722680 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:28.724173 (+ 1493us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:28.724373 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 19:47:28.724573 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 19:47:28.727077 23232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:28.739526 23232 catalog_manager.cc:1347] Generated new cluster ID: 39953898c1b54e198fc59bc1751ade9a
I20260807 19:47:28.739660 23232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:28.774055 23232 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:28.775425 23232 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:28.788134 23232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b8438ded618475e87b29ae551405fda: Generated new TSK 0
I20260807 19:47:28.789000 23232 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:28.801776 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 23135
I20260807 19:47:28.803064 23205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:28.901100 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 23135
W20260807 19:47:28.918661 23118 connection.cc:535] client connection to 127.30.208.62:43725 recv error: Network error: failed to read from TLS socket (remote: 127.30.208.62:43725): Connection reset by peer (error 104)
2026-08-07T19:47:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2085 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:29 dist-test-slave-vq2q krb5kdc[23247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:29 dist-test-slave-vq2q krb5kdc[23247](info): set up 2 sockets
Aug 07 19:47:29 dist-test-slave-vq2q krb5kdc[23247](info): commencing operation
krb5kdc: starting...
W20260807 19:47:30.004958 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.002s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:30 dist-test-slave-vq2q krb5kdc[23247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:47:30.313282 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:37279
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:34157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:37279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:30.653409 23261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:30.653715 23261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:30.653841 23261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:30.661206 23261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:30.661326 23261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:30.661396 23261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:30.661477 23261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:30.661537 23261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:30.671558 23261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:34157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:37279
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:37279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:30.672907 23261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:30.674290 23261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:30.686321 23272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:30.686398 23274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:30.686364 23271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:30.687254 23261 server_base.cc:1034] running on GCE node
I20260807 19:47:30.688143 23261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:30.689726 23261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:30.690970 23261 hybrid_clock.cc:648] HybridClock initialized: now 1786132050690926 us; error 42 us; skew 500 ppm
I20260807 19:47:30.691319 23261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:30 dist-test-slave-vq2q krb5kdc[23247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132050, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:30.694555 23261 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:47:30.696084 23261 webserver.cc:460] Webserver started at http://127.30.208.62:45403/ using document root <none> and password file <none>
I20260807 19:47:30.696735 23261 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:30.696837 23261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:30.697104 23261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:30.700214 23261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "20f11dcb06cf42868f2462dd6eb7cfca"
format_stamp: "Formatted at 2026-08-07 19:47:30 on dist-test-slave-vq2q"
server_key: "4b18e99f0236288f243e72f792067856"
server_key_iv: "881600892815e1377c796008fbe14a6a"
server_key_version: "encryptionkey@0"
I20260807 19:47:30.700968 23261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "20f11dcb06cf42868f2462dd6eb7cfca"
format_stamp: "Formatted at 2026-08-07 19:47:30 on dist-test-slave-vq2q"
server_key: "4b18e99f0236288f243e72f792067856"
server_key_iv: "881600892815e1377c796008fbe14a6a"
server_key_version: "encryptionkey@0"
I20260807 19:47:30.706445 23261 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 19:47:30.710094 23281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:30.711483 23261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:47:30.711630 23261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "20f11dcb06cf42868f2462dd6eb7cfca"
format_stamp: "Formatted at 2026-08-07 19:47:30 on dist-test-slave-vq2q"
server_key: "4b18e99f0236288f243e72f792067856"
server_key_iv: "881600892815e1377c796008fbe14a6a"
server_key_version: "encryptionkey@0"
I20260807 19:47:30.711803 23261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:30.755861 23261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:30.760200 23261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:30.760442 23261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:30.778992 23261 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:37279
I20260807 19:47:30.779011 23334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:37279 every 8 connection(s)
I20260807 19:47:30.780371 23261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:30.783790 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 23261
I20260807 19:47:30.783965 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:30.784278 23336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:30.784439 31552 external_mini_cluster.cc:1428] Setting key 6132c3b5281c02a50e1458ddb82c527c
I20260807 19:47:30.798009 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.786131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54123 (local address 127.30.208.62:37279)
0807 19:47:30.786908 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:30.786920 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:30.786964 (+ 44us) server_negotiation.cc:407] Connection header received
0807 19:47:30.788873 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:30.788895 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:30.789324 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:30.789641 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:30.790238 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.791739 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.793575 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.794253 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.794387 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.795631 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:30.795638 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:30.795655 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:30.796246 (+ 591us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:30.796324 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:30.796354 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:30.796480 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:30.796744 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 19:47:30.796966 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":348,"threads_started":1}
I20260807 19:47:30.798372 23336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca: Bootstrap starting.
I20260807 19:47:30.801934 23336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:30.802168 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:30.803001 23336 log.cc:824] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:30.805846 23336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca: No bootstrap required, opened a new log
I20260807 19:47:30.813509 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:30.806567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33435 (local address 127.30.208.62:37279)
0807 19:47:30.806746 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:30.806752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:30.807234 (+ 482us) server_negotiation.cc:407] Connection header received
0807 19:47:30.807382 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:30.807387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:30.807471 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:30.807578 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:30.808118 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.809300 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.810665 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:30.811410 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:30.811543 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:30.812650 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:30.812656 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:30.812662 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:30.812684 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:30.812731 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:30.812750 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:30.812867 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:30.813076 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 19:47:30.813138 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 19:47:30.815369 23336 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f11dcb06cf42868f2462dd6eb7cfca" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37279 } }
I20260807 19:47:30.815732 23336 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:30.815809 23336 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20f11dcb06cf42868f2462dd6eb7cfca, State: Initialized, Role: FOLLOWER
I20260807 19:47:30.816362 23336 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f11dcb06cf42868f2462dd6eb7cfca" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37279 } }
I20260807 19:47:30.816606 23336 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:30.816742 23336 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:30.816905 23336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:30.819186 23336 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f11dcb06cf42868f2462dd6eb7cfca" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37279 } }
I20260807 19:47:30.819690 23336 leader_election.cc:302] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20f11dcb06cf42868f2462dd6eb7cfca; no voters:
I20260807 19:47:30.820317 23336 leader_election.cc:288] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:30.820454 23348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:30.821345 23348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [term 1 LEADER]: Becoming Leader. State: Replica: 20f11dcb06cf42868f2462dd6eb7cfca, State: Running, Role: LEADER
I20260807 19:47:30.821869 23348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f11dcb06cf42868f2462dd6eb7cfca" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37279 } }
I20260807 19:47:30.822422 23336 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:30.824854 23350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20f11dcb06cf42868f2462dd6eb7cfca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f11dcb06cf42868f2462dd6eb7cfca" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37279 } } }
I20260807 19:47:30.824841 23351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20f11dcb06cf42868f2462dd6eb7cfca. Latest consensus state: current_term: 1 leader_uuid: "20f11dcb06cf42868f2462dd6eb7cfca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20f11dcb06cf42868f2462dd6eb7cfca" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 37279 } } }
I20260807 19:47:30.825330 23350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:30.825358 23351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:30.826171 23357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:30.832223 23357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:30.841265 23357 catalog_manager.cc:1347] Generated new cluster ID: cac3b72ee5d24ef78679617b56d7ad17
I20260807 19:47:30.841387 23357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:30.871905 23357 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:30.873952 23357 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:30.889101 23357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20f11dcb06cf42868f2462dd6eb7cfca: Generated new TSK 0
I20260807 19:47:30.890326 23357 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:30.897039 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 23261
W20260807 19:47:30.898572 23330 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54123
W20260807 19:47:30.899410 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 23261: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:47:30.899555 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 23261
W20260807 19:47:30.917842 23245 connection.cc:535] client connection to 127.30.208.62:37279 recv error: Network error: failed to read from TLS socket (remote: 127.30.208.62:37279): Connection reset by peer (error 104)
2026-08-07T19:47:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1998 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:31 dist-test-slave-vq2q krb5kdc[23370](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:31 dist-test-slave-vq2q krb5kdc[23370](info): set up 2 sockets
Aug 07 19:47:31 dist-test-slave-vq2q krb5kdc[23370](info): commencing operation
krb5kdc: starting...
W20260807 19:47:31.996161 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.006s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:32 dist-test-slave-vq2q krb5kdc[23370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:47:32.305413 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:42209
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:41653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:42209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:32.656723 23385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:32.657011 23385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:32.657138 23385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:32.664450 23385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:32.664561 23385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:32.664638 23385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:32.664721 23385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:32.664770 23385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:32.674724 23385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:41653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:42209
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:42209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:32.676033 23385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:32.677376 23385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:32.689358 23390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:32.689287 23391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:32.689329 23393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:32.690043 23385 server_base.cc:1034] running on GCE node
I20260807 19:47:32.690828 23385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:32.692329 23385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:32.693581 23385 hybrid_clock.cc:648] HybridClock initialized: now 1786132052693520 us; error 56 us; skew 500 ppm
I20260807 19:47:32.693970 23385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:32 dist-test-slave-vq2q krb5kdc[23370](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132052, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:32.697302 23385 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:47:32.698939 23385 webserver.cc:460] Webserver started at http://127.30.208.62:41905/ using document root <none> and password file <none>
I20260807 19:47:32.699674 23385 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:32.699805 23385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:32.700127 23385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:32.703315 23385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "8a6e2d72bbff4b93bf9ceb7d3c57dd95"
format_stamp: "Formatted at 2026-08-07 19:47:32 on dist-test-slave-vq2q"
server_key: "043680b1c2b8ad9990c68a673ba0f1d2"
server_key_iv: "f52d29a645e8ef3698c33f46a410f574"
server_key_version: "encryptionkey@0"
I20260807 19:47:32.704101 23385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "8a6e2d72bbff4b93bf9ceb7d3c57dd95"
format_stamp: "Formatted at 2026-08-07 19:47:32 on dist-test-slave-vq2q"
server_key: "043680b1c2b8ad9990c68a673ba0f1d2"
server_key_iv: "f52d29a645e8ef3698c33f46a410f574"
server_key_version: "encryptionkey@0"
I20260807 19:47:32.709647 23385 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:47:32.713338 23401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:32.714648 23385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 19:47:32.714813 23385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "8a6e2d72bbff4b93bf9ceb7d3c57dd95"
format_stamp: "Formatted at 2026-08-07 19:47:32 on dist-test-slave-vq2q"
server_key: "043680b1c2b8ad9990c68a673ba0f1d2"
server_key_iv: "f52d29a645e8ef3698c33f46a410f574"
server_key_version: "encryptionkey@0"
I20260807 19:47:32.715010 23385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:32.721081 23385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:32.725569 23385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:32.725876 23385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:32.744117 23385 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:42209
I20260807 19:47:32.744148 23461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:42209 every 8 connection(s)
I20260807 19:47:32.745574 23385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:32.749377 23462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:32.755615 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 23385
I20260807 19:47:32.755822 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:32.756364 31552 external_mini_cluster.cc:1428] Setting key 2e1caa9be89287b3baeca04d118adbf8
I20260807 19:47:32.761473 23462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95: Bootstrap starting.
I20260807 19:47:32.764847 23462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:32.764955 23466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:32.758483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45179 (local address 127.30.208.62:42209)
0807 19:47:32.759371 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:32.759383 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:32.759433 (+ 50us) server_negotiation.cc:407] Connection header received
0807 19:47:32.761154 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:32.761205 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:32.761638 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:32.762046 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:32.763340 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:32.763347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:32.763360 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:32.763375 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:32.763451 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:32.763470 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:32.763681 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:32.763830 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 19:47:32.764017 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":421,"threads_started":1}
I20260807 19:47:32.766227 23462 log.cc:824] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:32.769076 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:32.769412 23462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95: No bootstrap required, opened a new log
I20260807 19:47:32.775796 23466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:32.773380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44829 (local address 127.30.208.62:42209)
0807 19:47:32.773571 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:32.773578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:32.773901 (+ 323us) server_negotiation.cc:407] Connection header received
0807 19:47:32.774067 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:32.774072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:32.774139 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:32.774248 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:32.775129 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:32.775135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:32.775140 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:32.775167 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:32.775215 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:32.775235 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:32.775363 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:32.775414 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 19:47:32.775481 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 19:47:32.778004 23462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6e2d72bbff4b93bf9ceb7d3c57dd95" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 42209 } }
I20260807 19:47:32.778398 23462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:32.778503 23462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6e2d72bbff4b93bf9ceb7d3c57dd95, State: Initialized, Role: FOLLOWER
I20260807 19:47:32.779089 23462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6e2d72bbff4b93bf9ceb7d3c57dd95" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 42209 } }
I20260807 19:47:32.779327 23462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:32.779460 23462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:32.779611 23462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:32.781934 23462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6e2d72bbff4b93bf9ceb7d3c57dd95" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 42209 } }
I20260807 19:47:32.782438 23462 leader_election.cc:302] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a6e2d72bbff4b93bf9ceb7d3c57dd95; no voters:
I20260807 19:47:32.782991 23462 leader_election.cc:288] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:32.783108 23474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:32.783749 23474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6e2d72bbff4b93bf9ceb7d3c57dd95, State: Running, Role: LEADER
I20260807 19:47:32.784265 23474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6e2d72bbff4b93bf9ceb7d3c57dd95" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 42209 } }
I20260807 19:47:32.785009 23462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:32.786999 23475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a6e2d72bbff4b93bf9ceb7d3c57dd95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6e2d72bbff4b93bf9ceb7d3c57dd95" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 42209 } } }
I20260807 19:47:32.787488 23475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:32.787040 23476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a6e2d72bbff4b93bf9ceb7d3c57dd95. Latest consensus state: current_term: 1 leader_uuid: "8a6e2d72bbff4b93bf9ceb7d3c57dd95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6e2d72bbff4b93bf9ceb7d3c57dd95" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 42209 } } }
I20260807 19:47:32.787674 23476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:32.789857 23481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:32.795097 23481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:32.803258 23481 catalog_manager.cc:1347] Generated new cluster ID: 695df9cb7624440abeab6ba688394595
I20260807 19:47:32.803381 23481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:32.821600 23481 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:32.822094 23481 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:32.829456 23481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a6e2d72bbff4b93bf9ceb7d3c57dd95: Generated new TSK 0
I20260807 19:47:32.830410 23481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:32.846390 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 23385
W20260807 19:47:32.847504 23455 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45179
W20260807 19:47:32.848166 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 23385: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:47:32.848290 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 23385
2026-08-07T19:47:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1948 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:32 dist-test-slave-vq2q krb5kdc[23497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:32 dist-test-slave-vq2q krb5kdc[23497](info): set up 2 sockets
Aug 07 19:47:32 dist-test-slave-vq2q krb5kdc[23497](info): commencing operation
krb5kdc: starting...
W20260807 19:47:33.951803 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:34 dist-test-slave-vq2q krb5kdc[23497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:47:34.260776 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:40525
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:40525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:34.599923 23510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:34.600219 23510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:34.600368 23510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:34.607493 23510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:34.607615 23510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:34.607697 23510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:34.607788 23510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:34.607863 23510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:34.617533 23510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:40525
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:40525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:34.618839 23510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:34.620138 23510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:34.631551 23515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:34.631614 23516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:34.631721 23519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:34.637868 23510 server_base.cc:1034] running on GCE node
I20260807 19:47:34.638729 23510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:34.640233 23510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:34.641433 23510 hybrid_clock.cc:648] HybridClock initialized: now 1786132054641385 us; error 44 us; skew 500 ppm
I20260807 19:47:34.641788 23510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:34 dist-test-slave-vq2q krb5kdc[23497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132054, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:34.645241 23510 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:47:34.646804 23510 webserver.cc:460] Webserver started at http://127.30.208.62:33769/ using document root <none> and password file <none>
I20260807 19:47:34.647559 23510 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:34.647686 23510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:34.647991 23510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:34.651249 23510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "3827f5cda42641c68b2970a360e243c8"
format_stamp: "Formatted at 2026-08-07 19:47:34 on dist-test-slave-vq2q"
server_key: "8f624a60b5abffc1219d596a263a206a"
server_key_iv: "d7fdcc4efc6eedee43b2fd90f444253e"
server_key_version: "encryptionkey@0"
I20260807 19:47:34.652021 23510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "3827f5cda42641c68b2970a360e243c8"
format_stamp: "Formatted at 2026-08-07 19:47:34 on dist-test-slave-vq2q"
server_key: "8f624a60b5abffc1219d596a263a206a"
server_key_iv: "d7fdcc4efc6eedee43b2fd90f444253e"
server_key_version: "encryptionkey@0"
I20260807 19:47:34.657398 23510 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 19:47:34.660810 23527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:34.662231 23510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:47:34.662410 23510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "3827f5cda42641c68b2970a360e243c8"
format_stamp: "Formatted at 2026-08-07 19:47:34 on dist-test-slave-vq2q"
server_key: "8f624a60b5abffc1219d596a263a206a"
server_key_iv: "d7fdcc4efc6eedee43b2fd90f444253e"
server_key_version: "encryptionkey@0"
I20260807 19:47:34.662587 23510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:34.684713 23510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:34.689393 23510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:34.689669 23510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:34.708283 23510 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:40525
I20260807 19:47:34.708309 23583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:40525 every 8 connection(s)
I20260807 19:47:34.709677 23510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:34.711809 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 23510
I20260807 19:47:34.711970 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:34.712388 31552 external_mini_cluster.cc:1428] Setting key a548604a9f81d5eb0bb773400c100a40
I20260807 19:47:34.713716 23584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 19:47:34 dist-test-slave-vq2q krb5kdc[23497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132054, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:47:34.727980 23584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8: Bootstrap starting.
I20260807 19:47:34.731345 23584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:34.732383 23584 log.cc:824] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:34.735833 23584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8: No bootstrap required, opened a new log
I20260807 19:47:34.736020 23587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.714194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52259 (local address 127.30.208.62:40525)
0807 19:47:34.714956 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:34.714968 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:34.715008 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:47:34.716712 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:34.716732 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:34.717123 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:34.717491 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:34.719298 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.720649 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.722522 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.723282 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.723417 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:34.728239 (+ 4822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:34.728248 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:34.728269 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:34.728313 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:34.731313 (+ 3000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:34.732024 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:34.732032 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:34.732038 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:34.732210 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:34.732628 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:34.732634 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:34.732637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:34.733394 (+ 757us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:34.733636 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:34.734204 (+ 568us) server_negotiation.cc:299] Negotiation successful
0807 19:47:34.734646 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":336,"threads_started":1}
I20260807 19:47:34.739781 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:34.744004 23584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3827f5cda42641c68b2970a360e243c8" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40525 } }
I20260807 19:47:34.744513 23584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:34.744632 23584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3827f5cda42641c68b2970a360e243c8, State: Initialized, Role: FOLLOWER
I20260807 19:47:34.745352 23584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3827f5cda42641c68b2970a360e243c8" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40525 } }
I20260807 19:47:34.745601 23584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:34.745761 23584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:34.745914 23584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:34.748765 23584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3827f5cda42641c68b2970a360e243c8" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40525 } }
I20260807 19:47:34.749385 23584 leader_election.cc:302] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3827f5cda42641c68b2970a360e243c8; no voters:
I20260807 19:47:34.750100 23584 leader_election.cc:288] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:34.750344 23595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:34.752125 23584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:34.753471 23595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [term 1 LEADER]: Becoming Leader. State: Replica: 3827f5cda42641c68b2970a360e243c8, State: Running, Role: LEADER
I20260807 19:47:34.754063 23595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3827f5cda42641c68b2970a360e243c8" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40525 } }
I20260807 19:47:34.757076 23596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3827f5cda42641c68b2970a360e243c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3827f5cda42641c68b2970a360e243c8" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40525 } } }
I20260807 19:47:34.757551 23596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:34.761159 23587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:34.744035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49575 (local address 127.30.208.62:40525)
0807 19:47:34.744183 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:34.744188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:34.744778 (+ 590us) server_negotiation.cc:407] Connection header received
0807 19:47:34.745071 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:34.745076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:34.745157 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:34.745285 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:34.746819 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.747940 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.749259 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:34.749891 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:34.750027 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:34.753302 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:34.753310 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:34.753315 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:34.753365 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:34.757584 (+ 4219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:34.758614 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:34.758620 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:34.758625 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:34.758872 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:34.759386 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:34.759391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:34.759395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:34.759821 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:34.760128 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:34.760509 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 19:47:34.760771 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 19:47:34.762395 23597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3827f5cda42641c68b2970a360e243c8. Latest consensus state: current_term: 1 leader_uuid: "3827f5cda42641c68b2970a360e243c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3827f5cda42641c68b2970a360e243c8" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 40525 } } }
I20260807 19:47:34.762698 23597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:34.766662 23602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:34.770906 23602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:34.779980 23602 catalog_manager.cc:1347] Generated new cluster ID: ceea3480856446edaa23b1670af564d2
I20260807 19:47:34.780097 23602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:34.789776 23602 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:34.791494 23602 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:34.801692 23602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3827f5cda42641c68b2970a360e243c8: Generated new TSK 0
I20260807 19:47:34.802858 23602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:34.816120 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 23510
I20260807 19:47:34.817821 23579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:34.908597 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 23510
2026-08-07T19:47:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2059 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:35 dist-test-slave-vq2q krb5kdc[23619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:35 dist-test-slave-vq2q krb5kdc[23619](info): set up 2 sockets
Aug 07 19:47:35 dist-test-slave-vq2q krb5kdc[23619](info): commencing operation
krb5kdc: starting...
W20260807 19:47:35.992273 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:36 dist-test-slave-vq2q krb5kdc[23619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132056, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:47:36.298623 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33057
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:40583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:33057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:36.632490 23638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:36.632800 23638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:36.632896 23638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:36.640249 23638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:36.640370 23638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:36.640424 23638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:36.640484 23638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:36.640527 23638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:36.650636 23638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:40583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:33057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:33057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:36.651993 23638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:36.653321 23638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:36.666972 23646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:36.667111 23644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:36.667346 23643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:36.668151 23638 server_base.cc:1034] running on GCE node
I20260807 19:47:36.668959 23638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:36.670596 23638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:36.671835 23638 hybrid_clock.cc:648] HybridClock initialized: now 1786132056671777 us; error 46 us; skew 500 ppm
I20260807 19:47:36.672207 23638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:36 dist-test-slave-vq2q krb5kdc[23619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132056, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:36.675426 23638 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:47:36.677042 23638 webserver.cc:460] Webserver started at http://127.30.208.62:33227/ using document root <none> and password file <none>
I20260807 19:47:36.677750 23638 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:36.677863 23638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:36.678165 23638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:36.681227 23638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "5ee72c7f82304876b4f919f19d24c313"
format_stamp: "Formatted at 2026-08-07 19:47:36 on dist-test-slave-vq2q"
server_key: "c1d9ac66b1d40f1e53271877f29f4e3c"
server_key_iv: "24549f90459391be5c9379d96fabf079"
server_key_version: "encryptionkey@0"
I20260807 19:47:36.682031 23638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "5ee72c7f82304876b4f919f19d24c313"
format_stamp: "Formatted at 2026-08-07 19:47:36 on dist-test-slave-vq2q"
server_key: "c1d9ac66b1d40f1e53271877f29f4e3c"
server_key_iv: "24549f90459391be5c9379d96fabf079"
server_key_version: "encryptionkey@0"
I20260807 19:47:36.688392 23638 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 19:47:36.692358 23654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:36.693830 23638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:47:36.694016 23638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "5ee72c7f82304876b4f919f19d24c313"
format_stamp: "Formatted at 2026-08-07 19:47:36 on dist-test-slave-vq2q"
server_key: "c1d9ac66b1d40f1e53271877f29f4e3c"
server_key_iv: "24549f90459391be5c9379d96fabf079"
server_key_version: "encryptionkey@0"
I20260807 19:47:36.694214 23638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:36.719713 23638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:36.724184 23638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:36.724462 23638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:36.742157 23638 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:33057
I20260807 19:47:36.742180 23709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:33057 every 8 connection(s)
I20260807 19:47:36.743551 23638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:36.747246 23710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:36.747840 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 23638
I20260807 19:47:36.748013 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:36.748466 31552 external_mini_cluster.cc:1428] Setting key ebf3864c9bfe2534790d325dd8b56416
I20260807 19:47:36.761699 23710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313: Bootstrap starting.
I20260807 19:47:36.763851 23713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:36.750454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53655 (local address 127.30.208.62:33057)
0807 19:47:36.751385 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:36.751400 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:36.751772 (+ 372us) server_negotiation.cc:407] Connection header received
0807 19:47:36.753596 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:36.753627 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:36.754151 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:36.754615 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:36.755137 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:36.756995 (+ 1858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:36.758359 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:36.759425 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:36.759597 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:36.760800 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:36.760812 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:36.760831 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:36.761593 (+ 762us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:36.761700 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:36.761739 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:36.761857 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:36.762253 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 19:47:36.762539 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":367,"threads_started":1}
I20260807 19:47:36.765643 23710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:36.766793 23710 log.cc:824] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:36.767618 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:36.769488 23710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313: No bootstrap required, opened a new log
I20260807 19:47:36.778846 23713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:36.771906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53567 (local address 127.30.208.62:33057)
0807 19:47:36.772081 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:36.772086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:36.772599 (+ 513us) server_negotiation.cc:407] Connection header received
0807 19:47:36.772814 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:36.772817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:36.772880 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:36.772960 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:36.773609 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:36.774403 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:36.776313 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:36.776834 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:36.776944 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:36.778152 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:36.778157 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:36.778160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:36.778182 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:36.778221 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:36.778237 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:36.778308 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:36.778464 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 19:47:36.778546 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 19:47:36.778999 23710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee72c7f82304876b4f919f19d24c313" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33057 } }
I20260807 19:47:36.779484 23710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:36.779585 23710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ee72c7f82304876b4f919f19d24c313, State: Initialized, Role: FOLLOWER
I20260807 19:47:36.780172 23710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee72c7f82304876b4f919f19d24c313" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33057 } }
I20260807 19:47:36.780436 23710 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:36.780572 23710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:36.780731 23710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:36.782967 23710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee72c7f82304876b4f919f19d24c313" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33057 } }
I20260807 19:47:36.783555 23710 leader_election.cc:302] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ee72c7f82304876b4f919f19d24c313; no voters:
I20260807 19:47:36.784107 23710 leader_election.cc:288] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:36.784263 23721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:36.784893 23721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [term 1 LEADER]: Becoming Leader. State: Replica: 5ee72c7f82304876b4f919f19d24c313, State: Running, Role: LEADER
I20260807 19:47:36.785454 23721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee72c7f82304876b4f919f19d24c313" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33057 } }
I20260807 19:47:36.786187 23710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:36.788224 23722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ee72c7f82304876b4f919f19d24c313" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee72c7f82304876b4f919f19d24c313" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33057 } } }
I20260807 19:47:36.788434 23723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ee72c7f82304876b4f919f19d24c313. Latest consensus state: current_term: 1 leader_uuid: "5ee72c7f82304876b4f919f19d24c313" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ee72c7f82304876b4f919f19d24c313" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 33057 } } }
I20260807 19:47:36.788697 23722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:36.788769 23723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:36.790467 23727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:36.796674 23727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:36.808910 23727 catalog_manager.cc:1347] Generated new cluster ID: f72073ab83b34d3392ee0f2c968bc717
I20260807 19:47:36.809109 23727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:36.831247 23727 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:36.832580 23727 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:36.843648 23727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ee72c7f82304876b4f919f19d24c313: Generated new TSK 0
I20260807 19:47:36.844547 23727 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:36.857766 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 23638
W20260807 19:47:36.859318 23705 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53655
W20260807 19:47:36.859998 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 23638: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:47:36.860139 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 23638
2026-08-07T19:47:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1951 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:36 dist-test-slave-vq2q krb5kdc[23746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:36 dist-test-slave-vq2q krb5kdc[23746](info): set up 2 sockets
Aug 07 19:47:36 dist-test-slave-vq2q krb5kdc[23746](info): commencing operation
krb5kdc: starting...
W20260807 19:47:37.948253 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:38 dist-test-slave-vq2q krb5kdc[23746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:47:38.257119 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:41735
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:44177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:41735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:38.595198 23762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:38.595496 23762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:38.595635 23762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:38.602869 23762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:38.602998 23762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:38.603055 23762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:38.603139 23762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:38.603200 23762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:38.613287 23762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:44177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:41735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:41735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:38.614588 23762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:38.615945 23762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:38.627314 23769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:38.627333 23770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:38.627408 23772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:38.628046 23762 server_base.cc:1034] running on GCE node
I20260807 19:47:38.628857 23762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:38.630354 23762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:38.631558 23762 hybrid_clock.cc:648] HybridClock initialized: now 1786132058631515 us; error 40 us; skew 500 ppm
I20260807 19:47:38.631907 23762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:38 dist-test-slave-vq2q krb5kdc[23746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132058, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:38.638095 23762 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:47:38.639652 23762 webserver.cc:460] Webserver started at http://127.30.208.62:46101/ using document root <none> and password file <none>
I20260807 19:47:38.640359 23762 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:38.640482 23762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:38.640774 23762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:38.643896 23762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "801aca67962c4845bcd7f1e9245fc1eb"
format_stamp: "Formatted at 2026-08-07 19:47:38 on dist-test-slave-vq2q"
server_key: "0cb991085a1997fa0b9aa7c0ccf11fe3"
server_key_iv: "32d98f7339a22e3144b33dee147c224e"
server_key_version: "encryptionkey@0"
I20260807 19:47:38.644685 23762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "801aca67962c4845bcd7f1e9245fc1eb"
format_stamp: "Formatted at 2026-08-07 19:47:38 on dist-test-slave-vq2q"
server_key: "0cb991085a1997fa0b9aa7c0ccf11fe3"
server_key_iv: "32d98f7339a22e3144b33dee147c224e"
server_key_version: "encryptionkey@0"
I20260807 19:47:38.650079 23762 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 19:47:38.653750 23781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:38.655155 23762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 19:47:38.655355 23762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "801aca67962c4845bcd7f1e9245fc1eb"
format_stamp: "Formatted at 2026-08-07 19:47:38 on dist-test-slave-vq2q"
server_key: "0cb991085a1997fa0b9aa7c0ccf11fe3"
server_key_iv: "32d98f7339a22e3144b33dee147c224e"
server_key_version: "encryptionkey@0"
I20260807 19:47:38.655618 23762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:38.661607 23762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:38.665970 23762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:38.666265 23762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:38.684765 23762 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:41735
I20260807 19:47:38.684849 23840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:41735 every 8 connection(s)
I20260807 19:47:38.686509 23762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:38.687749 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 23762
I20260807 19:47:38.687935 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:38.688503 31552 external_mini_cluster.cc:1428] Setting key 2693bb227033bdd021b08deae6db35c9
I20260807 19:47:38.691697 23841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:38.697896 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:38.690512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37057 (local address 127.30.208.62:41735)
0807 19:47:38.691367 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:38.691383 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:38.691433 (+ 50us) server_negotiation.cc:407] Connection header received
0807 19:47:38.693768 (+ 2335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:38.693798 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:38.694230 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:38.694683 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:38.695832 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:38.695841 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:38.695858 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:38.695881 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:38.695983 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:38.696011 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:38.696260 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:38.696417 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 19:47:38.696625 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":341,"threads_started":1}
I20260807 19:47:38.701676 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:38.706295 23841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb: Bootstrap starting.
I20260807 19:47:38.709537 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:38.706973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43793 (local address 127.30.208.62:41735)
0807 19:47:38.707152 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:38.707158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:38.707628 (+ 470us) server_negotiation.cc:407] Connection header received
0807 19:47:38.707771 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:38.707777 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:38.707847 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:38.707981 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:38.708858 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:38.708864 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:38.708868 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 19:47:38.708887 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:38.708938 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 19:47:38.708958 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:38.709078 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:38.709152 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 19:47:38.709251 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 19:47:38.710212 23841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:38.711490 23841 log.cc:824] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:38.714552 23841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb: No bootstrap required, opened a new log
I20260807 19:47:38.722273 23841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "801aca67962c4845bcd7f1e9245fc1eb" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41735 } }
I20260807 19:47:38.722663 23841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 19:47:38.722739 23841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801aca67962c4845bcd7f1e9245fc1eb, State: Initialized, Role: FOLLOWER
I20260807 19:47:38.723269 23841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "801aca67962c4845bcd7f1e9245fc1eb" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41735 } }
I20260807 19:47:38.723474 23841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 19:47:38.723577 23841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 19:47:38.723704 23841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:38.726431 23841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "801aca67962c4845bcd7f1e9245fc1eb" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41735 } }
I20260807 19:47:38.726922 23841 leader_election.cc:302] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 801aca67962c4845bcd7f1e9245fc1eb; no voters:
I20260807 19:47:38.727473 23841 leader_election.cc:288] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 19:47:38.727619 23853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:38.728360 23853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [term 1 LEADER]: Becoming Leader. State: Replica: 801aca67962c4845bcd7f1e9245fc1eb, State: Running, Role: LEADER
I20260807 19:47:38.728879 23853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "801aca67962c4845bcd7f1e9245fc1eb" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41735 } }
I20260807 19:47:38.729521 23841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:38.731604 23854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "801aca67962c4845bcd7f1e9245fc1eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "801aca67962c4845bcd7f1e9245fc1eb" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41735 } } }
I20260807 19:47:38.731593 23855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 801aca67962c4845bcd7f1e9245fc1eb. Latest consensus state: current_term: 1 leader_uuid: "801aca67962c4845bcd7f1e9245fc1eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "801aca67962c4845bcd7f1e9245fc1eb" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 41735 } } }
I20260807 19:47:38.732163 23854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:38.732240 23855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:38.736410 23860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:38.740731 23860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:38.750046 23860 catalog_manager.cc:1347] Generated new cluster ID: 92a0df4dfa7d4f59968a139f35bbba05
I20260807 19:47:38.750180 23860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:38.773276 23860 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:38.773805 23860 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:38.783296 23860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 801aca67962c4845bcd7f1e9245fc1eb: Generated new TSK 0
I20260807 19:47:38.784395 23860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:38.791654 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 23762
W20260807 19:47:38.793051 23836 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37057
W20260807 19:47:38.793758 31552 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 23762: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 19:47:38.793912 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 23762
2026-08-07T19:47:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1933 ms)
[----------] 18 tests from AuthTokenIssuingTest (35870 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:38 dist-test-slave-vq2q krb5kdc[23877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:38 dist-test-slave-vq2q krb5kdc[23877](info): set up 2 sockets
Aug 07 19:47:38 dist-test-slave-vq2q krb5kdc[23877](info): commencing operation
krb5kdc: starting...
W20260807 19:47:39.897254 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:40 dist-test-slave-vq2q krb5kdc[23877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:47:40.208324 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:36825
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:36825,127.30.208.61:37507,127.30.208.60:44727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:40.544953 23890 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:47:40.545306 23890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:40.545419 23890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:40.545542 23890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:40.552919 23890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:40.553020 23890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:40.553081 23890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:40.553149 23890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:40.553248 23890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:40.563316 23890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:36825,127.30.208.61:37507,127.30.208.60:44727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:36825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:40.564670 23890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:40.566073 23890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:40.577508 23895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:40.577560 23899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:40.577585 23896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:40.578317 23890 server_base.cc:1034] running on GCE node
I20260807 19:47:40.579136 23890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:40.580525 23890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:40.581749 23890 hybrid_clock.cc:648] HybridClock initialized: now 1786132060581679 us; error 47 us; skew 500 ppm
I20260807 19:47:40.582113 23890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:40 dist-test-slave-vq2q krb5kdc[23877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:40.585201 23890 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:47:40.586625 23890 webserver.cc:460] Webserver started at http://127.30.208.62:42475/ using document root <none> and password file <none>
I20260807 19:47:40.587306 23890 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:40.587409 23890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:40.587718 23890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:40.590708 23890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "50f4d28abc0145438f6a805289b5d026"
format_stamp: "Formatted at 2026-08-07 19:47:40 on dist-test-slave-vq2q"
server_key: "db793fbd54ed7d162580c69556fbbbab"
server_key_iv: "b357036475a2a95ceab8b1d2267af985"
server_key_version: "encryptionkey@0"
I20260807 19:47:40.591475 23890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "50f4d28abc0145438f6a805289b5d026"
format_stamp: "Formatted at 2026-08-07 19:47:40 on dist-test-slave-vq2q"
server_key: "db793fbd54ed7d162580c69556fbbbab"
server_key_iv: "b357036475a2a95ceab8b1d2267af985"
server_key_version: "encryptionkey@0"
I20260807 19:47:40.596793 23890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260807 19:47:40.600263 23908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:40.601675 23890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 19:47:40.601835 23890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "50f4d28abc0145438f6a805289b5d026"
format_stamp: "Formatted at 2026-08-07 19:47:40 on dist-test-slave-vq2q"
server_key: "db793fbd54ed7d162580c69556fbbbab"
server_key_iv: "b357036475a2a95ceab8b1d2267af985"
server_key_version: "encryptionkey@0"
I20260807 19:47:40.602043 23890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:40.629129 23890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:40.633426 23890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:40.633694 23890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:40.651547 23890 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:36825
I20260807 19:47:40.651569 23965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:36825 every 8 connection(s)
I20260807 19:47:40.652880 23890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:40.656500 23966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:40.659492 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 23890
I20260807 19:47:40.659685 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:40.660303 31552 external_mini_cluster.cc:1428] Setting key f15315977ec7573c0faaecbf7cd19181
I20260807 19:47:40.663058 23966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:40 dist-test-slave-vq2q krb5kdc[23877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:47:40.689297 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:40.666232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:36825 (local address 127.0.0.1:54692)
0807 19:47:40.670126 (+ 3894us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:40.670169 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:40.672000 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:40.672978 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:40.672989 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:40.673686 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:40.674007 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:40.674024 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:40.675632 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:40.675637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:40.677151 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:40.677167 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:40.678096 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:40.678106 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:40.678268 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:40.679047 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:40.679074 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:40.681684 (+ 2610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:40.684870 (+ 3186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:40.684882 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:40.684896 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:40.685359 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:40.685771 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:40.685776 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:40.685780 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:40.685939 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:40.686727 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:40.686735 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:40.686963 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 19:47:40.687191 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 19:47:40.687890 (+ 699us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3637,"mutex_wait_us":184,"spinlock_wait_cycles":42752,"thread_start_us":3432,"threads_started":1}
I20260807 19:47:40.689328 23968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:40.666232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54692 (local address 127.30.208.62:36825)
0807 19:47:40.669682 (+ 3450us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:40.669695 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:40.670312 (+ 617us) server_negotiation.cc:407] Connection header received
0807 19:47:40.672374 (+ 2062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:40.672382 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:40.672668 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:40.672891 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:40.674236 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:40.675463 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:40.677315 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:40.677857 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:40.677995 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:40.681927 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:40.681935 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:40.681948 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:40.681978 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:40.684718 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:40.685492 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:40.685499 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:40.685501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:40.685652 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:40.686057 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:40.686060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:40.686062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:40.686588 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:40.686795 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:40.687342 (+ 547us) server_negotiation.cc:299] Negotiation successful
0807 19:47:40.687770 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":3223,"thread_start_us":2987,"threads_started":1}
WARNING: no policy specified for kudu/127.30.208.61@KRBTEST.COM; defaulting to no policy
I20260807 19:47:40.692220 23966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.61@KRBTEST.COM" created.
I20260807 19:47:40.693337 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:40.692905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37507 (local address 127.0.0.1:55410)
0807 19:47:40.693078 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:40.693215 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 19:47:40.693604 23909 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
I20260807 19:47:40.695055 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:40.694726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37507 (local address 127.0.0.1:55422)
0807 19:47:40.694860 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:40.694930 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 19:47:40.695250 23966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:37507: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.61.keytab.
Entry for principal kudu/127.30.208.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.61.keytab.
I20260807 19:47:40.729074 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.61:37507
--webserver_interface=127.30.208.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.61.keytab
--principal=kudu/127.30.208.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:36825,127.30.208.61:37507,127.30.208.60:44727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:47:40.744622 23966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } attempt: 1
I20260807 19:47:40.745873 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:40.745418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37507 (local address 127.0.0.1:55434)
0807 19:47:40.745607 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:40.745738 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:47:40.746799 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:40.746461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37507 (local address 127.0.0.1:55436)
0807 19:47:40.746601 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:40.746668 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 19:47:40.746970 23966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:37507: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
I20260807 19:47:40.815227 23966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } attempt: 2
I20260807 19:47:40.816476 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:40.815963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37507 (local address 127.0.0.1:55448)
0807 19:47:40.816215 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:40.816320 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 19:47:40.817492 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:40.817098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37507 (local address 127.0.0.1:55450)
0807 19:47:40.817281 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:40.817354 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 19:47:40.817664 23966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:37507: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
I20260807 19:47:40.908929 23966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } attempt: 3
I20260807 19:47:40.910053 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:40.909625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37507 (local address 127.0.0.1:55466)
0807 19:47:40.909820 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:40.909918 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 19:47:40.910971 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:40.910621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37507 (local address 127.0.0.1:55470)
0807 19:47:40.910762 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:40.910831 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 19:47:40.911149 23966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:37507: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
I20260807 19:47:41.054402 23966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } attempt: 4
I20260807 19:47:41.055626 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.055072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37507 (local address 127.0.0.1:55482)
0807 19:47:41.055391 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.055485 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260807 19:47:41.056794 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.056358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37507 (local address 127.0.0.1:55494)
0807 19:47:41.056524 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.056634 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 19:47:41.057022 23966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:37507: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37507: connect: Connection refused (error 111)
W20260807 19:47:41.068876 23973 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:47:41.069195 23973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:41.069293 23973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:41.069386 23973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:41.076740 23973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:41.076830 23973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:41.076895 23973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:41.076967 23973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:41.077054 23973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:41.087179 23973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:36825,127.30.208.61:37507,127.30.208.60:44727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.61.keytab
--principal=kudu/127.30.208.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.61:37507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.208.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:41.088457 23973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:41.089874 23973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:41.101401 23980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:41.101488 23983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:41.101450 23981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:41.102196 23973 server_base.cc:1034] running on GCE node
I20260807 19:47:41.102989 23973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:41.104391 23973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:41.105612 23973 hybrid_clock.cc:648] HybridClock initialized: now 1786132061105544 us; error 46 us; skew 500 ppm
I20260807 19:47:41.105976 23973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:41 dist-test-slave-vq2q krb5kdc[23877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:41.109112 23973 init.cc:377] Logged in from keytab as kudu/127.30.208.61@KRBTEST.COM (short username kudu)
I20260807 19:47:41.110625 23973 webserver.cc:460] Webserver started at http://127.30.208.61:44005/ using document root <none> and password file <none>
I20260807 19:47:41.111295 23973 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:41.111397 23973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:41.111711 23973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:41.114709 23973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/data/instance:
uuid: "6dd8a68d849940a29b1434f02d458bfb"
format_stamp: "Formatted at 2026-08-07 19:47:41 on dist-test-slave-vq2q"
server_key: "263c09f6642d634261c9fea5bb516ac8"
server_key_iv: "50bf9ead329b3c8404586a7cd737683e"
server_key_version: "encryptionkey@0"
I20260807 19:47:41.115522 23973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/wal/instance:
uuid: "6dd8a68d849940a29b1434f02d458bfb"
format_stamp: "Formatted at 2026-08-07 19:47:41 on dist-test-slave-vq2q"
server_key: "263c09f6642d634261c9fea5bb516ac8"
server_key_iv: "50bf9ead329b3c8404586a7cd737683e"
server_key_version: "encryptionkey@0"
I20260807 19:47:41.120543 23973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:47:41.123905 23991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:41.125211 23973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 19:47:41.125372 23973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/wal
uuid: "6dd8a68d849940a29b1434f02d458bfb"
format_stamp: "Formatted at 2026-08-07 19:47:41 on dist-test-slave-vq2q"
server_key: "263c09f6642d634261c9fea5bb516ac8"
server_key_iv: "50bf9ead329b3c8404586a7cd737683e"
server_key_version: "encryptionkey@0"
I20260807 19:47:41.125577 23973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:41.145637 23973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:41.149899 23973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:41.150152 23973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:41.167657 23973 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.61:37507
I20260807 19:47:41.167677 24048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.61:37507 every 8 connection(s)
I20260807 19:47:41.168951 23973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/data/info.pb
I20260807 19:47:41.170327 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 23973
I20260807 19:47:41.170503 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-1/wal/instance
I20260807 19:47:41.171052 31552 external_mini_cluster.cc:1428] Setting key 0c1623dc4e0749684be3d48f917b40e2
I20260807 19:47:41.173372 24051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:41.182003 24051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:41 dist-test-slave-vq2q krb5kdc[23877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.61@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.208.60@KRBTEST.COM; defaulting to no policy
I20260807 19:47:41.202410 23968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.185318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54704 (local address 127.30.208.62:36825)
0807 19:47:41.185518 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:41.185523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:41.186783 (+ 1260us) server_negotiation.cc:407] Connection header received
0807 19:47:41.188732 (+ 1949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:41.188739 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:41.188821 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:41.188909 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:41.190511 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.191254 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.192804 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.193240 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.193363 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.196669 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:41.196674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:41.196677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:41.196710 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:41.199363 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.200098 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.200104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.200107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.200232 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.200639 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.200644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.200648 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.200939 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:41.201090 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:41.201841 (+ 751us) server_negotiation.cc:299] Negotiation successful
0807 19:47:41.202081 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 19:47:41.203418 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.185837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:36825 (local address 127.0.0.1:54704)
0807 19:47:41.186612 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.186656 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:41.188434 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:41.189156 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:41.189164 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:41.189733 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:41.190323 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.190342 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.191427 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.191431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.192651 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.192664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.193460 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.193467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.193697 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.194323 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:41.194344 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:41.196450 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:41.199504 (+ 3054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.199513 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.199532 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.199983 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.200339 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.200344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.200348 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.200537 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.201068 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:41.201076 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:41.201466 (+ 390us) client_negotiation.cc:769] Sending connection context
0807 19:47:41.201721 (+ 255us) client_negotiation.cc:240] Negotiation successful
0807 19:47:41.202239 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":321,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.60@KRBTEST.COM" created.
I20260807 19:47:41.205375 24051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:41 dist-test-slave-vq2q krb5kdc[23877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.61@KRBTEST.COM for kudu/127.30.208.61@KRBTEST.COM
I20260807 19:47:41.220809 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.206227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37507 (local address 127.0.0.1:55506)
0807 19:47:41.206448 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.206469 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:41.206566 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:41.207573 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:41.207577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:41.208007 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:41.208172 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.208182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.209595 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.209600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.210710 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.210725 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.211578 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.211583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.211736 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.212392 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:41.212413 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:41.214932 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:41.218223 (+ 3291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.218228 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.218232 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.218686 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.219101 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.219106 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.219109 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.219275 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.219919 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:41.219924 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:41.220061 (+ 137us) client_negotiation.cc:769] Sending connection context
0807 19:47:41.220198 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 19:47:41.220469 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 19:47:41.220929 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.206484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55506 (local address 127.30.208.61:37507)
0807 19:47:41.207144 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:41.207150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:41.207185 (+ 35us) server_negotiation.cc:407] Connection header received
0807 19:47:41.207229 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:41.207235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:41.207418 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:41.207528 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:41.208330 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.209468 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.210867 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.211429 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.211536 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.215216 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:41.215235 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:41.215255 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:41.215304 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:41.218092 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.218832 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.218840 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.218842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.218985 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.219388 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.219391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.219393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.219770 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:41.219947 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:41.220386 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 19:47:41.220603 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":360,"threads_started":1}
I20260807 19:47:41.223840 24051 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:47:41.225008 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.224486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41396)
0807 19:47:41.224668 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.224809 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 19:47:41.225332 23995 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
I20260807 19:47:41.226917 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.226547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41410)
0807 19:47:41.226702 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.226794 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 19:47:41.227167 24051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:44727: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.60.keytab.
Entry for principal kudu/127.30.208.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.60.keytab.
I20260807 19:47:41.242450 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.60:44727
--webserver_interface=127.30.208.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.60.keytab
--principal=kudu/127.30.208.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:36535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:36825,127.30.208.61:37507,127.30.208.60:44727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:47:41.276544 24051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } attempt: 1
I20260807 19:47:41.277909 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.277347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41422)
0807 19:47:41.277583 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.277708 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 19:47:41.279055 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.278592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41428)
0807 19:47:41.278738 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.278910 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 19:47:41.279284 24051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:44727: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
I20260807 19:47:41.347527 24051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } attempt: 2
I20260807 19:47:41.348763 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.348218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41442)
0807 19:47:41.348452 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.348567 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 19:47:41.349876 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.349473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41452)
0807 19:47:41.349632 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.349725 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 19:47:41.350080 24051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:44727: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
I20260807 19:47:41.356290 23966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } attempt: 5
Aug 07 19:47:41 dist-test-slave-vq2q krb5kdc[23877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for kudu/127.30.208.61@KRBTEST.COM
I20260807 19:47:41.369796 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.356999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37507 (local address 127.0.0.1:55518)
0807 19:47:41.357234 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.357255 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:41.357399 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:41.357755 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:41.357759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:41.358170 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:41.358350 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.358360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.359338 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.359343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.360210 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.360222 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.360943 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.360948 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.361153 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.361741 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:41.361759 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:41.363933 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:41.367579 (+ 3646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.367583 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.367586 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.367952 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.368311 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.368314 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.368317 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.368458 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.369015 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:41.369018 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:41.369156 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 19:47:41.369335 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 19:47:41.369558 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 19:47:41.369976 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.357060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55518 (local address 127.30.208.61:37507)
0807 19:47:41.357244 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:41.357250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:41.357329 (+ 79us) server_negotiation.cc:407] Connection header received
0807 19:47:41.357530 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:41.357534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:41.357615 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:41.357699 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:41.358495 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.359201 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.360357 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.360801 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.360917 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.364137 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:41.364144 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:41.364148 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:41.364189 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:41.367415 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.368059 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.368064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.368067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.368208 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.368549 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.368554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.368557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.368913 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:41.369055 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:41.369459 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 19:47:41.369686 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 19:47:41.370853 23966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:47:41.371915 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.371471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41454)
0807 19:47:41.371643 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.371739 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 19:47:41.372825 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.372450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41466)
0807 19:47:41.372608 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.372696 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 19:47:41.373097 23966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:44727: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
I20260807 19:47:41.424413 23966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } attempt: 1
I20260807 19:47:41.425835 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.425221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41480)
0807 19:47:41.425467 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.425595 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 19:47:41.427170 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.426710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41482)
0807 19:47:41.426921 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.427027 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 19:47:41.427410 23966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:44727: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
I20260807 19:47:41.441291 24051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } attempt: 3
I20260807 19:47:41.442591 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.442000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41486)
0807 19:47:41.442244 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.442388 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 19:47:41.443832 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.443264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41488)
0807 19:47:41.443556 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.443664 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175}
W20260807 19:47:41.444097 24051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:44727: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
I20260807 19:47:41.495652 23966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } attempt: 2
I20260807 19:47:41.496958 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.496398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41498)
0807 19:47:41.496621 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.496734 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 19:47:41.498064 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.497668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41506)
0807 19:47:41.497819 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.497921 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 19:47:41.498270 23966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:44727: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
W20260807 19:47:41.581431 24058 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:47:41.581751 24058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:41.581820 24058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:41.581910 24058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 19:47:41.587352 24051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } attempt: 4
I20260807 19:47:41.588590 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.588055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41522)
0807 19:47:41.588295 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.588408 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 19:47:41.589633 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.589271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41534)
0807 19:47:41.589423 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.589503 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 19:47:41.589644 24058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:41.589726 24058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:41.589808 24058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:41.589891 24058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:41.589919 24051 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:44727: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
W20260807 19:47:41.589978 24058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:41.599970 24058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:36535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:36825,127.30.208.61:37507,127.30.208.60:44727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.60.keytab
--principal=kudu/127.30.208.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.60:44727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.208.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:41.601322 24058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:41.602586 24058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 19:47:41.604528 23966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } attempt: 3
I20260807 19:47:41.605830 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.605153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41538)
0807 19:47:41.605565 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.605662 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238}
I20260807 19:47:41.606853 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.606487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41546)
0807 19:47:41.606667 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.606746 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260807 19:47:41.607059 23966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:44727: Network error: Client connection negotiation failed: client connection to 127.30.208.60:44727: connect: Connection refused (error 111)
W20260807 19:47:41.614645 24064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:41.614625 24063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:41.614606 24066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:41.618667 24058 server_base.cc:1034] running on GCE node
I20260807 19:47:41.619484 24058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:41.621006 24058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:41.622241 24058 hybrid_clock.cc:648] HybridClock initialized: now 1786132061622171 us; error 48 us; skew 500 ppm
I20260807 19:47:41.622542 24058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:41 dist-test-slave-vq2q krb5kdc[23877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:41.625648 24058 init.cc:377] Logged in from keytab as kudu/127.30.208.60@KRBTEST.COM (short username kudu)
I20260807 19:47:41.627146 24058 webserver.cc:460] Webserver started at http://127.30.208.60:32857/ using document root <none> and password file <none>
I20260807 19:47:41.627795 24058 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:41.627889 24058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:41.628152 24058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:41.631131 24058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/data/instance:
uuid: "da9cf1c781844eb289a7621bcbf53ead"
format_stamp: "Formatted at 2026-08-07 19:47:41 on dist-test-slave-vq2q"
server_key: "a7298aa4ed399477a809f326b2de7195"
server_key_iv: "0cd31ef550df3166cec71bbd05727b11"
server_key_version: "encryptionkey@0"
I20260807 19:47:41.631867 24058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/wal/instance:
uuid: "da9cf1c781844eb289a7621bcbf53ead"
format_stamp: "Formatted at 2026-08-07 19:47:41 on dist-test-slave-vq2q"
server_key: "a7298aa4ed399477a809f326b2de7195"
server_key_iv: "0cd31ef550df3166cec71bbd05727b11"
server_key_version: "encryptionkey@0"
I20260807 19:47:41.636780 24058 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 19:47:41.640049 24074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:41.641453 24058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:47:41.641624 24058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/wal
uuid: "da9cf1c781844eb289a7621bcbf53ead"
format_stamp: "Formatted at 2026-08-07 19:47:41 on dist-test-slave-vq2q"
server_key: "a7298aa4ed399477a809f326b2de7195"
server_key_iv: "0cd31ef550df3166cec71bbd05727b11"
server_key_version: "encryptionkey@0"
I20260807 19:47:41.641844 24058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:41.667426 24058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:41.671382 24058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:41.671634 24058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:41.689471 24058 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.60:44727
I20260807 19:47:41.689497 24130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.60:44727 every 8 connection(s)
I20260807 19:47:41.690873 24058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/data/info.pb
I20260807 19:47:41.692274 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 24058
I20260807 19:47:41.692488 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786131702042267-31552-0/minicluster-data/master-2/wal/instance
I20260807 19:47:41.693034 31552 external_mini_cluster.cc:1428] Setting key 8d03a08ec713be5d8223d90c98f45bbf
I20260807 19:47:41.695411 24131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:41.702679 24131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:41 dist-test-slave-vq2q krb5kdc[23877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.60@KRBTEST.COM
Aug 07 19:47:41 dist-test-slave-vq2q krb5kdc[23877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.60@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:47:41.715386 24135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.695063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41552 (local address 127.30.208.60:44727)
0807 19:47:41.695897 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:41.695909 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:41.695949 (+ 40us) server_negotiation.cc:407] Connection header received
0807 19:47:41.697744 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:41.697766 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:41.698144 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:41.698506 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:41.699993 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.701369 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.702593 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.703410 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.703578 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.708235 (+ 4657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:41.708242 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:41.708252 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:41.708291 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:41.711144 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.711947 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.711955 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.711963 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.712185 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.712622 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.712625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.712628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.713149 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:41.713471 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:41.713955 (+ 484us) server_negotiation.cc:299] Negotiation successful
0807 19:47:41.714342 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":359,"threads_started":1}
I20260807 19:47:41.719590 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.705549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:36825 (local address 127.0.0.1:54708)
0807 19:47:41.706451 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.706494 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:41.706653 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:41.707061 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:41.707066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:41.707684 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:41.707939 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.707953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.709078 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.709082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.710252 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.710265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.710987 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.710994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.711142 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.711684 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:41.711708 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:41.713988 (+ 2280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:41.716990 (+ 3002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.717001 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.717008 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.717467 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.717839 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.717843 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.717845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.717965 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.718627 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:41.718634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:41.718829 (+ 195us) client_negotiation.cc:769] Sending connection context
0807 19:47:41.719014 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 19:47:41.719307 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":525,"threads_started":1}
I20260807 19:47:41.720796 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.705495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54708 (local address 127.30.208.62:36825)
0807 19:47:41.706179 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:41.706185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:41.706593 (+ 408us) server_negotiation.cc:407] Connection header received
0807 19:47:41.706779 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:41.706784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:41.706871 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:41.706996 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:41.708094 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.708910 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.710395 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.710787 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.710903 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.714174 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:41.714181 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:41.714185 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:41.714224 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:41.716867 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.717588 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.717593 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.717596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.717726 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.718109 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.718114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.718117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.718490 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:41.718640 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:41.720304 (+ 1664us) server_negotiation.cc:299] Negotiation successful
0807 19:47:41.720483 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":327,"threads_started":1}
I20260807 19:47:41.722218 24131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:41 dist-test-slave-vq2q krb5kdc[23877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
Aug 07 19:47:41 dist-test-slave-vq2q krb5kdc[23877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.60@KRBTEST.COM for kudu/127.30.208.61@KRBTEST.COM
I20260807 19:47:41.732446 24139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.718137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54720 (local address 127.30.208.62:36825)
0807 19:47:41.718774 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:41.718780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:41.718802 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:47:41.718861 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:41.718866 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:41.718933 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:41.719025 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:41.720224 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.721250 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.722521 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.723059 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.723175 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.727006 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:41.727012 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:41.727017 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:41.727053 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:41.729886 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.730560 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.730566 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.730569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.730694 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.731045 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.731050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.731054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.731328 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:41.731574 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:41.731887 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 19:47:41.732100 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":313,"threads_started":1}
I20260807 19:47:41.735260 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.722998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37507 (local address 127.0.0.1:55532)
0807 19:47:41.723325 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.723346 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:41.723439 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:41.723734 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:41.723738 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:41.724082 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:41.724213 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.724221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.725136 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.725142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.726068 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.726080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.726755 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.726761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.726959 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.727463 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:41.727486 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:41.729738 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:41.732708 (+ 2970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.732713 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.732716 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.733166 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.733756 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.733760 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.733762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.733956 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.734582 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:41.734585 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:41.734716 (+ 131us) client_negotiation.cc:769] Sending connection context
0807 19:47:41.734815 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 19:47:41.735030 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207}
I20260807 19:47:41.735821 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.723000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55532 (local address 127.30.208.61:37507)
0807 19:47:41.723178 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:41.723183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:41.723408 (+ 225us) server_negotiation.cc:407] Connection header received
0807 19:47:41.723554 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:41.723559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:41.723624 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:41.723699 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:41.724349 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.725010 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.726203 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.726618 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.726721 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.729950 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:41.729955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:41.729958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:41.729992 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:41.732550 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.733463 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.733467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.733469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.733648 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.734084 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.734089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.734092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.734434 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:41.734606 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:41.734945 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 19:47:41.735186 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 19:47:41.736699 24131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:41 dist-test-slave-vq2q krb5kdc[23877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.60@KRBTEST.COM for kudu/127.30.208.60@KRBTEST.COM
Aug 07 19:47:41 dist-test-slave-vq2q krb5kdc[23877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.61@KRBTEST.COM
I20260807 19:47:41.749953 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.737407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41560)
0807 19:47:41.737685 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.737701 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:41.737786 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:41.738088 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:41.738092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:41.738525 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:41.738694 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.738705 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.739816 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.739820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.740784 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.740798 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.741637 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.741642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.741908 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.742356 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:41.742372 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:41.744277 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:41.747652 (+ 3375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.747656 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.747658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.748008 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.748391 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.748395 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.748398 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.748547 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.749049 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:41.749052 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:41.749230 (+ 178us) client_negotiation.cc:769] Sending connection context
0807 19:47:41.749407 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 19:47:41.749654 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186}
I20260807 19:47:41.750213 24135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.737674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41560 (local address 127.30.208.60:44727)
0807 19:47:41.737839 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:41.737844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:41.737862 (+ 18us) server_negotiation.cc:407] Connection header received
0807 19:47:41.737908 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:41.737913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:41.737977 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:41.738077 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:41.738829 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.739687 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.740930 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.741485 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.741609 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.744452 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:41.744456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:41.744459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:41.744488 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:41.747507 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.748118 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.748123 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.748126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.748273 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.748647 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.748650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.748652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.748933 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:41.749076 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:41.749510 (+ 434us) server_negotiation.cc:299] Negotiation successful
0807 19:47:41.749856 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":56}
I20260807 19:47:41.751128 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.734136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55544 (local address 127.30.208.61:37507)
0807 19:47:41.736319 (+ 2183us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:41.736326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:41.736341 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:47:41.736386 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:41.736391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:41.736462 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:41.736560 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:41.738130 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.739085 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.740456 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.741049 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.741134 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.745261 (+ 4127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:41.745268 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:41.745272 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:41.745305 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:41.748158 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.748896 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.748901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.748905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.749066 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.749460 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.749465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.749468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.749784 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:41.750063 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:41.750599 (+ 536us) server_negotiation.cc:299] Negotiation successful
0807 19:47:41.750806 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2082,"thread_start_us":3207,"threads_started":1}
I20260807 19:47:41.755461 24131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead: Bootstrap starting.
I20260807 19:47:41.755740 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:41.758769 24131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:41.759811 24131 log.cc:824] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:41.758383 23938 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "50f4d28abc0145438f6a805289b5d026"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54720
I20260807 19:47:41.762737 24131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead: No bootstrap required, opened a new log
I20260807 19:47:41.769869 24131 raft_consensus.cc:348] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } }
I20260807 19:47:41.770200 24131 raft_consensus.cc:729] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da9cf1c781844eb289a7621bcbf53ead, State: Initialized, Role: FOLLOWER
I20260807 19:47:41.770627 24131 consensus_queue.cc:260] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } }
I20260807 19:47:41.771459 24143 sys_catalog.cc:455] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } } }
I20260807 19:47:41.771762 24143 sys_catalog.cc:458] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:41.772706 24131 sys_catalog.cc:564] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:41.784226 23966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } attempt: 4
W20260807 19:47:41.785410 24155 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:47:41.785542 24155 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 19:47:41.783529 24058 master_runner.cc:186] Error getting master registration for 127.30.208.62:36825: OK, instance_id { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" instance_seqno: 1786132060646509 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:47:41.794677 24058 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 07 19:47:41 dist-test-slave-vq2q krb5kdc[23877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for kudu/127.30.208.60@KRBTEST.COM
I20260807 19:47:41.804184 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.784948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41562)
0807 19:47:41.785127 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.785146 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:41.785290 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:41.789724 (+ 4434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:41.789730 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:41.790144 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:41.790309 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.790319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.791805 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.791818 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.792726 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.792738 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.793842 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.793848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.794030 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.794737 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:41.794761 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:41.797671 (+ 2910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:41.801299 (+ 3628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.801305 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.801310 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.801889 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.802289 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.802294 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.802298 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.802498 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.803114 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:41.803119 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:41.803324 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 19:47:41.803487 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 19:47:41.803790 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 19:47:41.804415 24135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.785290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41562 (local address 127.30.208.60:44727)
0807 19:47:41.789408 (+ 4118us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:41.789414 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:41.789438 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:47:41.789499 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:41.789505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:41.789579 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:41.789687 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:41.790470 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.791673 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.792868 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.793677 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.793815 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.797874 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:41.797881 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:41.797886 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:41.797926 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:41.801117 (+ 3191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.802025 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.802029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.802031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.802169 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.802600 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.802603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.802605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.802962 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:41.803088 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:41.803620 (+ 532us) server_negotiation.cc:299] Negotiation successful
0807 19:47:41.803917 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3940}
I20260807 19:47:41.810592 23966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026: Bootstrap starting.
I20260807 19:47:41.814990 23966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:41.816439 23966 log.cc:824] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:41.820065 23966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026: No bootstrap required, opened a new log
I20260807 19:47:41.828040 23966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } }
I20260807 19:47:41.828555 23966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50f4d28abc0145438f6a805289b5d026, State: Initialized, Role: FOLLOWER
I20260807 19:47:41.829298 23966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } }
I20260807 19:47:41.830431 24159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } } }
I20260807 19:47:41.830895 24159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:41.831832 23966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:41.831892 23938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:47:41.832077 23938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:41.834820 23938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } }
I20260807 19:47:41.836436 23938 leader_election.cc:288] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [CANDIDATE]: Term 1 election: Requested vote from peers 6dd8a68d849940a29b1434f02d458bfb (127.30.208.61:37507), da9cf1c781844eb289a7621bcbf53ead (127.30.208.60:44727)
I20260807 19:47:41.837677 24103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50f4d28abc0145438f6a805289b5d026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "da9cf1c781844eb289a7621bcbf53ead"
I20260807 19:47:41.838250 24103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:41.842236 24103 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50f4d28abc0145438f6a805289b5d026 in term 1.
I20260807 19:47:41.843031 23912 leader_election.cc:302] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50f4d28abc0145438f6a805289b5d026, da9cf1c781844eb289a7621bcbf53ead; no voters:
I20260807 19:47:41.843603 24159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:41.846506 24159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [term 1 LEADER]: Becoming Leader. State: Replica: 50f4d28abc0145438f6a805289b5d026, State: Running, Role: LEADER
I20260807 19:47:41.847239 24159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } }
I20260807 19:47:41.859652 24165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50f4d28abc0145438f6a805289b5d026. Latest consensus state: current_term: 1 leader_uuid: "50f4d28abc0145438f6a805289b5d026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } } }
I20260807 19:47:41.860129 24165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:41.861375 24171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 19:47:41.886174 24171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 19:47:41.886694 23914 tablet.cc:2369] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:47:41.889129 24051 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } attempt: 5
I20260807 19:47:41.904582 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.865329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55552 (local address 127.30.208.61:37507)
0807 19:47:41.865539 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:41.865546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:41.872676 (+ 7130us) server_negotiation.cc:407] Connection header received
0807 19:47:41.872996 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:41.873002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:41.873097 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:41.873189 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:41.875725 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.876923 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.881313 (+ 4390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.882051 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.882158 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.892983 (+ 10825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:41.892991 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:41.892997 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:41.893044 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:41.896824 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.901310 (+ 4486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.901317 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.901322 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.901534 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.902042 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.902048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.902052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.902457 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:41.902722 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:41.903893 (+ 1171us) server_negotiation.cc:299] Negotiation successful
0807 19:47:41.904172 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 19:47:41 dist-test-slave-vq2q krb5kdc[23877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.61@KRBTEST.COM for kudu/127.30.208.60@KRBTEST.COM
I20260807 19:47:41.923059 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.889724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:44727 (local address 127.0.0.1:41576)
0807 19:47:41.890116 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:41.890136 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:41.890255 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:41.907087 (+ 16832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:41.907092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:41.907579 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:41.907792 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.907805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.909695 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.909701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.911028 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:41.911045 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.912053 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.912059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:41.912337 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.912999 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:41.913023 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:41.915844 (+ 2821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:41.920116 (+ 4272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.920121 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.920125 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.920659 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.921148 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:41.921153 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:41.921157 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:41.921361 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:41.922013 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:41.922018 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:41.922213 (+ 195us) client_negotiation.cc:769] Sending connection context
0807 19:47:41.922359 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 19:47:41.922661 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266}
W20260807 19:47:41.923749 24081 tablet.cc:2369] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:47:41.925493 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.890469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41576 (local address 127.30.208.60:44727)
0807 19:47:41.906675 (+ 16206us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:41.906681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:41.906703 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:47:41.906750 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:41.906754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:41.906815 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:41.907955 (+ 1140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:41.908266 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.909538 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.911245 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.911864 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.912679 (+ 815us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.916112 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:41.916119 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:41.916122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:41.916178 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:41.919950 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.920829 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.920834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.920837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.921009 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.921505 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.921509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.921513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.921865 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:41.924664 (+ 2799us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:41.924814 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 19:47:41.925066 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16072,"thread_start_us":16418,"threads_started":1}
I20260807 19:47:41.927609 24139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.871757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54724 (local address 127.30.208.62:36825)
0807 19:47:41.871985 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:41.871993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:41.882605 (+ 10612us) server_negotiation.cc:407] Connection header received
0807 19:47:41.885413 (+ 2808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:41.885420 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:41.885512 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:41.885614 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:41.892321 (+ 6707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.893655 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.903249 (+ 9594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.904724 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.904855 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.913089 (+ 8234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:41.913099 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:41.913105 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:41.913164 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:41.917147 (+ 3983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.924538 (+ 7391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.924546 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.924550 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.924757 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.925291 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.925297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.925301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.925739 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:41.926023 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:41.926918 (+ 895us) server_negotiation.cc:299] Negotiation successful
0807 19:47:41.927218 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 19:47:41.933554 24051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb: Bootstrap starting.
I20260807 19:47:41.936664 24103 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead [term 1 FOLLOWER]: Refusing update from remote peer 50f4d28abc0145438f6a805289b5d026: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:41.937805 24165 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:41.938355 24135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:41.881724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41572 (local address 127.30.208.60:44727)
0807 19:47:41.881896 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:41.881902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:41.904211 (+ 22309us) server_negotiation.cc:407] Connection header received
0807 19:47:41.905384 (+ 1173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:41.905391 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:41.905485 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:41.905582 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:41.913300 (+ 7718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.914318 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.917573 (+ 3255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:41.918320 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:41.918429 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:41.928784 (+ 10355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:41.928791 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:41.928796 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:41.928857 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:41.933075 (+ 4218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.934858 (+ 1783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.934864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.934867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.935056 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:41.936434 (+ 1378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:41.936439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:41.936443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:41.936830 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:41.937097 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:41.937734 (+ 637us) server_negotiation.cc:299] Negotiation successful
0807 19:47:41.938004 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 19:47:41.946460 24051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:41.947851 24051 log.cc:824] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:41.942855 23890 master_runner.cc:186] Error getting master registration for 127.30.208.61:37507: OK, instance_id { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" instance_seqno: 1786132061162660 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:47:41.953017 23890 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:47:41.958592 24143 sys_catalog.cc:455] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50f4d28abc0145438f6a805289b5d026. Latest consensus state: current_term: 1 leader_uuid: "50f4d28abc0145438f6a805289b5d026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } } }
I20260807 19:47:41.958941 24143 sys_catalog.cc:458] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:41.962026 24051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb: No bootstrap required, opened a new log
I20260807 19:47:41.966949 24185 mvcc.cc:205] Tried to move back new op lower bound from 7315996925658865664 to 7315996925338415104. Current Snapshot: MvccSnapshot[applied={T|T < 7315996925658865664}]
I20260807 19:47:41.975696 24159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50f4d28abc0145438f6a805289b5d026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } } }
I20260807 19:47:41.976046 24159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:41.979060 24171 catalog_manager.cc:1347] Generated new cluster ID: 974d4b7b0ee64fd5b3a046f5dc978831
I20260807 19:47:41.979169 24171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:41.981302 24051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } }
I20260807 19:47:41.981900 24051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd8a68d849940a29b1434f02d458bfb, State: Initialized, Role: FOLLOWER
I20260807 19:47:41.982679 24051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } }
I20260807 19:47:41.984022 24191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } } }
I20260807 19:47:41.984658 24191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:41.986182 24021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50f4d28abc0145438f6a805289b5d026" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6dd8a68d849940a29b1434f02d458bfb"
I20260807 19:47:41.986510 24020 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:41.986923 24051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:41.990123 24020 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb [term 1 FOLLOWER]: Refusing update from remote peer 50f4d28abc0145438f6a805289b5d026: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:41.993716 24021 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50f4d28abc0145438f6a805289b5d026 in term 1.
I20260807 19:47:41.996007 24165 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:41.997246 24143 sys_catalog.cc:455] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50f4d28abc0145438f6a805289b5d026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } } }
I20260807 19:47:41.997520 24143 sys_catalog.cc:458] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:42.027753 24171 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:42.029814 24171 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:42.040264 24191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50f4d28abc0145438f6a805289b5d026. Latest consensus state: current_term: 1 leader_uuid: "50f4d28abc0145438f6a805289b5d026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } } }
I20260807 19:47:42.040661 24191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:42.040293 24197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50f4d28abc0145438f6a805289b5d026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } } }
I20260807 19:47:42.040773 24197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:42.045632 24159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50f4d28abc0145438f6a805289b5d026" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50f4d28abc0145438f6a805289b5d026" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 36825 } } peers { permanent_uuid: "6dd8a68d849940a29b1434f02d458bfb" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37507 } } peers { permanent_uuid: "da9cf1c781844eb289a7621bcbf53ead" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 44727 } } }
I20260807 19:47:42.045974 24159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:42.066735 24171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50f4d28abc0145438f6a805289b5d026: Generated new TSK 0
I20260807 19:47:42.067997 24171 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:42.098387 24208 catalog_manager.cc:1259] Loaded cluster ID: 974d4b7b0ee64fd5b3a046f5dc978831
I20260807 19:47:42.098573 24208 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb: loading cluster ID for follower catalog manager: success
I20260807 19:47:42.102435 24208 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb: acquiring CA information for follower catalog manager: success
I20260807 19:47:42.105249 24208 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6dd8a68d849940a29b1434f02d458bfb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 19:47:42.204439 31552 mini_kdc.cc:348] Time spent kdestroy: real 0.112s user 0.000s sys 0.028s
W20260807 19:47:42.257731 24055 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:42.243122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55562 (local address 127.30.208.61:37507)
0807 19:47:42.243375 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.243382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.255380 (+ 11998us) server_negotiation.cc:407] Connection header received
0807 19:47:42.255953 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.255959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.256055 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.256140 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:42.257434 (+ 1294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55562: BlockingRecv error: recv got EOF from 127.0.0.1:55562 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 19:47:42.260596 24139 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:42.251366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54732 (local address 127.30.208.62:36825)
0807 19:47:42.251572 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.251579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.258707 (+ 7128us) server_negotiation.cc:407] Connection header received
0807 19:47:42.258988 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.258994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.259089 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.259162 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:42.260293 (+ 1131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54732: BlockingRecv error: recv got EOF from 127.0.0.1:54732 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 19:47:42.264225 24180 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:42.262015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41578 (local address 127.30.208.60:44727)
0807 19:47:42.262208 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.262215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.262639 (+ 424us) server_negotiation.cc:407] Connection header received
0807 19:47:42.262899 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.262904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.262986 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.263062 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:42.264002 (+ 940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41578: BlockingRecv error: recv got EOF from 127.0.0.1:41578 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 19:47:42.317963 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.298473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55568 (local address 127.30.208.61:37507)
0807 19:47:42.298667 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.298686 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.309413 (+ 10727us) server_negotiation.cc:407] Connection header received
0807 19:47:42.310034 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.310042 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.310288 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.310455 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:42.311248 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.312423 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.314542 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.315209 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.315321 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:42.316157 (+ 836us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:42.317030 (+ 873us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:42.317169 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:42.317464 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 19:47:42.317566 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 19:47:42.366801 24180 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:42.357320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41590 (local address 127.30.208.60:44727)
0807 19:47:42.357513 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.357519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.364505 (+ 6986us) server_negotiation.cc:407] Connection header received
0807 19:47:42.365094 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.365100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.365258 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.365340 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:42.366507 (+ 1167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41590: BlockingRecv error: recv got EOF from 127.0.0.1:41590 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:47:42.407446 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.401639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55574 (local address 127.30.208.61:37507)
0807 19:47:42.401903 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.401910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.402412 (+ 502us) server_negotiation.cc:407] Connection header received
0807 19:47:42.402601 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.402607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.402681 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.402762 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:42.403211 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.403995 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.405488 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.406110 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.406249 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:42.406680 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:42.406822 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:42.406895 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:42.407039 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 19:47:42.407121 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 19:47:42.416524 24180 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:42.414484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41606 (local address 127.30.208.60:44727)
0807 19:47:42.414695 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.414702 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.415216 (+ 514us) server_negotiation.cc:407] Connection header received
0807 19:47:42.415398 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.415403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.415478 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.415557 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:42.416306 (+ 749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41606: BlockingRecv error: recv got EOF from 127.0.0.1:41606 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:47:42.426656 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.421077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55578 (local address 127.30.208.61:37507)
0807 19:47:42.421305 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.421312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.421798 (+ 486us) server_negotiation.cc:407] Connection header received
0807 19:47:42.421991 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.421997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.422064 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.422139 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:42.422567 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.423360 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.424783 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.425234 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.425354 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:42.425891 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:42.426016 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:42.426089 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:42.426232 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 19:47:42.426333 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 19:47:42.435132 24180 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:42.432803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41610 (local address 127.30.208.60:44727)
0807 19:47:42.432999 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.433007 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.433711 (+ 704us) server_negotiation.cc:407] Connection header received
0807 19:47:42.433909 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.433914 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.434004 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.434089 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:42.434901 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41610: BlockingRecv error: recv got EOF from 127.0.0.1:41610 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 19:47:42.452229 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.446734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55594 (local address 127.30.208.61:37507)
0807 19:47:42.447021 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.447028 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.447507 (+ 479us) server_negotiation.cc:407] Connection header received
0807 19:47:42.447704 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.447709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.447778 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.447865 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:42.448311 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.449087 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.450468 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.450963 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.451078 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:42.451493 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:42.451601 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:42.451683 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:42.451839 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 19:47:42.451923 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
W20260807 19:47:42.463099 24180 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:42.458154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41626 (local address 127.30.208.60:44727)
0807 19:47:42.458436 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.458443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.461705 (+ 3262us) server_negotiation.cc:407] Connection header received
0807 19:47:42.461871 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.461877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.461957 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.462039 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:42.462874 (+ 835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41626: BlockingRecv error: recv got EOF from 127.0.0.1:41626 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
I20260807 19:47:42.479725 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.473952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55610 (local address 127.30.208.61:37507)
0807 19:47:42.474156 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.474164 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.474701 (+ 537us) server_negotiation.cc:407] Connection header received
0807 19:47:42.474868 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.474874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.474953 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.475041 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:42.475460 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.476359 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.477863 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.478323 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.478432 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:42.478853 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:42.478970 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:42.479052 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:42.479318 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 19:47:42.479404 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 19:47:42.491104 24180 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:42.486087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41638 (local address 127.30.208.60:44727)
0807 19:47:42.486327 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.486333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.489751 (+ 3418us) server_negotiation.cc:407] Connection header received
0807 19:47:42.489915 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.489921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.490003 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.490087 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:42.490870 (+ 783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41638: BlockingRecv error: recv got EOF from 127.0.0.1:41638 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:47:42.516095 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.510201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55612 (local address 127.30.208.61:37507)
0807 19:47:42.510437 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.510444 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.510954 (+ 510us) server_negotiation.cc:407] Connection header received
0807 19:47:42.511158 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.511162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.511233 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.511312 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:42.511728 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.512587 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.513974 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.514529 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.514645 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:42.515278 (+ 633us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:42.515414 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:42.515478 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:42.515713 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 19:47:42.515794 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 19:47:42.524709 24180 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:42.522513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41644 (local address 127.30.208.60:44727)
0807 19:47:42.522679 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.522683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.523361 (+ 678us) server_negotiation.cc:407] Connection header received
0807 19:47:42.523542 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.523546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.523621 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.523703 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:42.524520 (+ 817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41644: BlockingRecv error: recv got EOF from 127.0.0.1:41644 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 19:47:42.566362 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.560484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55616 (local address 127.30.208.61:37507)
0807 19:47:42.560731 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.560739 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.561265 (+ 526us) server_negotiation.cc:407] Connection header received
0807 19:47:42.561484 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.561489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.561562 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.561640 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:42.562132 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.562980 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.564406 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.565063 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.565202 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:42.565611 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:42.565736 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:42.565818 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:42.566001 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 19:47:42.566090 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 19:47:42.574717 24180 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:42.572511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41646 (local address 127.30.208.60:44727)
0807 19:47:42.572741 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.572748 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.573334 (+ 586us) server_negotiation.cc:407] Connection header received
0807 19:47:42.573529 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.573533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.573603 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.573681 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:42.574476 (+ 795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41646: BlockingRecv error: recv got EOF from 127.0.0.1:41646 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 19:47:42.652660 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.647258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55624 (local address 127.30.208.61:37507)
0807 19:47:42.647482 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.647489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.647741 (+ 252us) server_negotiation.cc:407] Connection header received
0807 19:47:42.647907 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.647911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.647983 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.648059 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:42.648436 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.649266 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.650813 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.651406 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.651532 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:42.652001 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:42.652137 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:42.652219 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:42.652317 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 19:47:42.652395 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 19:47:42.661255 24180 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:42.659529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41656 (local address 127.30.208.60:44727)
0807 19:47:42.659725 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.659732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.660006 (+ 274us) server_negotiation.cc:407] Connection header received
0807 19:47:42.660160 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.660165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.660247 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.660357 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:42.661018 (+ 661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41656: BlockingRecv error: recv got EOF from 127.0.0.1:41656 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:47:42.787835 24155 catalog_manager.cc:1259] Loaded cluster ID: 974d4b7b0ee64fd5b3a046f5dc978831
I20260807 19:47:42.787954 24155 catalog_manager.cc:1552] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead: loading cluster ID for follower catalog manager: success
I20260807 19:47:42.790688 24155 catalog_manager.cc:1574] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead: acquiring CA information for follower catalog manager: success
I20260807 19:47:42.792689 24155 catalog_manager.cc:1602] T 00000000000000000000000000000000 P da9cf1c781844eb289a7621bcbf53ead: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:47:42.798326 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.793298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55626 (local address 127.30.208.61:37507)
0807 19:47:42.793519 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.793525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.793793 (+ 268us) server_negotiation.cc:407] Connection header received
0807 19:47:42.793961 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.793965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.794034 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.794111 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:42.794447 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.795185 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.796641 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.797099 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.797240 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:42.797622 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:42.797756 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:42.797835 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:42.797935 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 19:47:42.798009 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 19:47:42.810230 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.804325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41662 (local address 127.30.208.60:44727)
0807 19:47:42.804577 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.804584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.804877 (+ 293us) server_negotiation.cc:407] Connection header received
0807 19:47:42.805053 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.805057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.805262 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.805376 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:42.805846 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.806561 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.807892 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.808317 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.808433 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:42.808971 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:42.809582 (+ 611us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:42.809726 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:42.809838 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 19:47:42.809934 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 19:47:42.824414 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.817024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55640 (local address 127.30.208.61:37507)
0807 19:47:42.817264 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.817272 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.818247 (+ 975us) server_negotiation.cc:407] Connection header received
0807 19:47:42.818490 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.818497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.818591 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.818657 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:42.819402 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.820343 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.822189 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.822938 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.823099 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:42.823551 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:42.823778 (+ 227us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:42.823873 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:42.824061 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 19:47:42.824148 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 19:47:42.825162 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.818208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41670 (local address 127.30.208.60:44727)
0807 19:47:42.818400 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.818407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.818885 (+ 478us) server_negotiation.cc:407] Connection header received
0807 19:47:42.819033 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.819037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.819122 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.819199 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:42.819994 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.821023 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.822828 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.823472 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.823601 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:42.824064 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:42.824212 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:42.824294 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:42.824807 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 19:47:42.824901 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 19:47:42.825455 24327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:42.816929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54736 (local address 127.30.208.62:36825)
0807 19:47:42.817654 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:42.817661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:42.818095 (+ 434us) server_negotiation.cc:407] Connection header received
0807 19:47:42.818726 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:42.818730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:42.818926 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:42.819125 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:42.819798 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.820921 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.823001 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:42.823534 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:42.823651 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:42.824383 (+ 732us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:42.824830 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:42.824950 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:42.825054 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 19:47:42.825138 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":455,"threads_started":1}
I20260807 19:47:42.830448 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 23890
I20260807 19:47:42.831561 23961 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:42.942893 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 23890
I20260807 19:47:42.964506 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 23973
I20260807 19:47:42.965615 24043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:43.073415 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 23973
I20260807 19:47:43.093165 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 24058
I20260807 19:47:43.094389 24125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:43.202224 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 24058
2026-08-07T19:47:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4413 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 19:47:43 dist-test-slave-vq2q krb5kdc[24334](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 19:47:43 dist-test-slave-vq2q krb5kdc[24334](info): set up 2 sockets
Aug 07 19:47:43 dist-test-slave-vq2q krb5kdc[24334](info): commencing operation
krb5kdc: starting...
W20260807 19:47:44.312587 31552 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/test-user.keytab.
Aug 07 19:47:44 dist-test-slave-vq2q krb5kdc[24334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T19:47:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T19:47:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
Entry for principal kudu/127.30.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab.
I20260807 19:47:44.636457 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:34683
--webserver_interface=127.30.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:35295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:34683,127.30.208.61:37991,127.30.208.60:34203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 19:47:44.980717 24351 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:47:44.981061 24351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:44.981168 24351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:44.981315 24351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:44.988862 24351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:44.988966 24351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:44.989029 24351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:44.989101 24351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:44.989197 24351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:44.999393 24351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:35295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:34683,127.30.208.61:37991,127.30.208.60:34203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.62.keytab
--principal=kudu/127.30.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.62:34683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:45.000902 24351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:45.002288 24351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:45.013952 24359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:45.013919 24358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:45.014108 24361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:45.015230 24351 server_base.cc:1034] running on GCE node
I20260807 19:47:45.016021 24351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:45.017465 24351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:45.018711 24351 hybrid_clock.cc:648] HybridClock initialized: now 1786132065018632 us; error 42 us; skew 500 ppm
I20260807 19:47:45.019028 24351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:45 dist-test-slave-vq2q krb5kdc[24334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:45.025020 24351 init.cc:377] Logged in from keytab as kudu/127.30.208.62@KRBTEST.COM (short username kudu)
I20260807 19:47:45.026516 24351 webserver.cc:460] Webserver started at http://127.30.208.62:34701/ using document root <none> and password file <none>
I20260807 19:47:45.027217 24351 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:45.027321 24351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:45.027632 24351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:45.030670 24351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/data/instance:
uuid: "9c13d3955af84691b14a3bb3ce24a891"
format_stamp: "Formatted at 2026-08-07 19:47:45 on dist-test-slave-vq2q"
server_key: "2465f4c716a75da3e2b3b15b2005d2f8"
server_key_iv: "e7a9512f4c136e059e99e824768dd4af"
server_key_version: "encryptionkey@0"
I20260807 19:47:45.031479 24351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/wal/instance:
uuid: "9c13d3955af84691b14a3bb3ce24a891"
format_stamp: "Formatted at 2026-08-07 19:47:45 on dist-test-slave-vq2q"
server_key: "2465f4c716a75da3e2b3b15b2005d2f8"
server_key_iv: "e7a9512f4c136e059e99e824768dd4af"
server_key_version: "encryptionkey@0"
I20260807 19:47:45.036710 24351 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 19:47:45.040329 24368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:45.041801 24351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:47:45.041982 24351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/wal
uuid: "9c13d3955af84691b14a3bb3ce24a891"
format_stamp: "Formatted at 2026-08-07 19:47:45 on dist-test-slave-vq2q"
server_key: "2465f4c716a75da3e2b3b15b2005d2f8"
server_key_iv: "e7a9512f4c136e059e99e824768dd4af"
server_key_version: "encryptionkey@0"
I20260807 19:47:45.042197 24351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:45.060333 24351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:45.065958 24351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:45.066274 24351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:45.084995 24351 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.62:34683
I20260807 19:47:45.085019 24424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.62:34683 every 8 connection(s)
I20260807 19:47:45.086445 24351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/data/info.pb
I20260807 19:47:45.088194 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 24351
I20260807 19:47:45.088414 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-0/wal/instance
I20260807 19:47:45.088989 31552 external_mini_cluster.cc:1428] Setting key 0e4fdeed3c8d7789c8999b710a2ff8d2
I20260807 19:47:45.091212 24425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:45.100172 24425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:45 dist-test-slave-vq2q krb5kdc[24334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.208.61@KRBTEST.COM; defaulting to no policy
I20260807 19:47:45.124356 24429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.103957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44116 (local address 127.30.208.62:34683)
0807 19:47:45.105760 (+ 1803us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:45.105768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:45.105822 (+ 54us) server_negotiation.cc:407] Connection header received
0807 19:47:45.107394 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:45.107401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:45.107699 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:45.107914 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:45.109085 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.110823 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.112177 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.112992 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.113149 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:45.116582 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:45.116592 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:45.116607 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:45.116641 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:45.120118 (+ 3477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:45.120904 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:45.120914 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:45.120918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:45.121120 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:45.121566 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:45.121571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:45.121575 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:45.122282 (+ 707us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:45.122490 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:45.123020 (+ 530us) server_negotiation.cc:299] Negotiation successful
0807 19:47:45.123456 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":138,"server-negotiator.queue_time_us":1578,"spinlock_wait_cycles":25856,"thread_start_us":1362,"threads_started":1}
I20260807 19:47:45.124388 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.103957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34683 (local address 127.0.0.1:44116)
0807 19:47:45.104893 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.104959 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:45.107024 (+ 2065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:45.107935 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:45.107940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:45.108521 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:45.108867 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:45.108886 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.110973 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.110977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:45.112038 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:45.112051 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.113241 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.113247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:45.113386 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:45.114008 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:45.114030 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:45.116343 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:45.120280 (+ 3937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:45.120291 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:45.120301 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:45.120725 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:45.121270 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:45.121274 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:45.121276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:45.121456 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:45.122436 (+ 980us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:45.122444 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:45.122631 (+ 187us) client_negotiation.cc:769] Sending connection context
0807 19:47:45.122839 (+ 208us) client_negotiation.cc:240] Negotiation successful
0807 19:47:45.123298 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":378,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.61@KRBTEST.COM" created.
I20260807 19:47:45.127384 24425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:47:45.128625 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.128175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37991 (local address 127.0.0.1:54568)
0807 19:47:45.128368 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.128497 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 19:47:45.128921 24372 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
I20260807 19:47:45.130496 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.130108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37991 (local address 127.0.0.1:54582)
0807 19:47:45.130269 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.130348 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 19:47:45.130697 24425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:37991: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.61.keytab.
Entry for principal kudu/127.30.208.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.61.keytab.
I20260807 19:47:45.161087 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.61:37991
--webserver_interface=127.30.208.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.61.keytab
--principal=kudu/127.30.208.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:35295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:34683,127.30.208.61:37991,127.30.208.60:34203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:47:45.180096 24425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } attempt: 1
I20260807 19:47:45.181403 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.180861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37991 (local address 127.0.0.1:54592)
0807 19:47:45.181088 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.181254 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 19:47:45.182385 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.182019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37991 (local address 127.0.0.1:54602)
0807 19:47:45.182166 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.182244 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 19:47:45.182582 24425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:37991: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
I20260807 19:47:45.250818 24425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } attempt: 2
I20260807 19:47:45.251878 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.251421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37991 (local address 127.0.0.1:54608)
0807 19:47:45.251649 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.251738 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 19:47:45.252825 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.252467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37991 (local address 127.0.0.1:54610)
0807 19:47:45.252607 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.252694 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 19:47:45.253001 24425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:37991: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
I20260807 19:47:45.344252 24425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } attempt: 3
I20260807 19:47:45.345451 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.344993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37991 (local address 127.0.0.1:54622)
0807 19:47:45.345205 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.345314 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 19:47:45.346427 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.346083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37991 (local address 127.0.0.1:54630)
0807 19:47:45.346221 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.346297 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 19:47:45.346606 24425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:37991: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
I20260807 19:47:45.489845 24425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } attempt: 4
I20260807 19:47:45.490967 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.490462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37991 (local address 127.0.0.1:54638)
0807 19:47:45.490729 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.490833 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260807 19:47:45.491950 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.491601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37991 (local address 127.0.0.1:54640)
0807 19:47:45.491745 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.491828 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 19:47:45.492143 24425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.61:37991: Network error: Client connection negotiation failed: client connection to 127.30.208.61:37991: connect: Connection refused (error 111)
W20260807 19:47:45.495831 24432 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:47:45.496158 24432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:45.496258 24432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:45.496368 24432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:45.503664 24432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:45.503760 24432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:45.503852 24432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:45.503917 24432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:45.503999 24432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:45.513881 24432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:35295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:34683,127.30.208.61:37991,127.30.208.60:34203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.61.keytab
--principal=kudu/127.30.208.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.61:37991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.208.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:45.515146 24432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:45.516403 24432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:45.527904 24438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:45.527980 24442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:45.528009 24439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:45.529080 24432 server_base.cc:1034] running on GCE node
I20260807 19:47:45.529999 24432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:45.531549 24432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:45.532806 24432 hybrid_clock.cc:648] HybridClock initialized: now 1786132065532735 us; error 39 us; skew 500 ppm
I20260807 19:47:45.533154 24432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:45 dist-test-slave-vq2q krb5kdc[24334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:45.536419 24432 init.cc:377] Logged in from keytab as kudu/127.30.208.61@KRBTEST.COM (short username kudu)
I20260807 19:47:45.538019 24432 webserver.cc:460] Webserver started at http://127.30.208.61:41677/ using document root <none> and password file <none>
I20260807 19:47:45.538704 24432 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:45.538800 24432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:45.539076 24432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:45.542311 24432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/data/instance:
uuid: "411480b61aa14fc8b56ab4819c79bf85"
format_stamp: "Formatted at 2026-08-07 19:47:45 on dist-test-slave-vq2q"
server_key: "3261e9db0e7ef9359817667ea3d728d7"
server_key_iv: "dc487697300c33e0da79f81d16fbc0d6"
server_key_version: "encryptionkey@0"
I20260807 19:47:45.543398 24432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/wal/instance:
uuid: "411480b61aa14fc8b56ab4819c79bf85"
format_stamp: "Formatted at 2026-08-07 19:47:45 on dist-test-slave-vq2q"
server_key: "3261e9db0e7ef9359817667ea3d728d7"
server_key_iv: "dc487697300c33e0da79f81d16fbc0d6"
server_key_version: "encryptionkey@0"
I20260807 19:47:45.548830 24432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 19:47:45.552474 24451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:45.553881 24432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 19:47:45.554069 24432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/wal
uuid: "411480b61aa14fc8b56ab4819c79bf85"
format_stamp: "Formatted at 2026-08-07 19:47:45 on dist-test-slave-vq2q"
server_key: "3261e9db0e7ef9359817667ea3d728d7"
server_key_iv: "dc487697300c33e0da79f81d16fbc0d6"
server_key_version: "encryptionkey@0"
I20260807 19:47:45.554297 24432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:45.576995 24432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:45.581015 24432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:45.581310 24432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:45.600204 24432 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.61:37991
I20260807 19:47:45.600253 24507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.61:37991 every 8 connection(s)
I20260807 19:47:45.601568 24432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/data/info.pb
I20260807 19:47:45.603101 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 24432
I20260807 19:47:45.603284 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-1/wal/instance
I20260807 19:47:45.603899 31552 external_mini_cluster.cc:1428] Setting key 184bc3f12454d31fb23d4c5489fd02fd
I20260807 19:47:45.606145 24508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:45.614660 24508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:45 dist-test-slave-vq2q krb5kdc[24334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.61@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.208.60@KRBTEST.COM; defaulting to no policy
I20260807 19:47:45.634976 24429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.617828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.30.208.62:34683)
0807 19:47:45.618050 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:45.618056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:45.619385 (+ 1329us) server_negotiation.cc:407] Connection header received
0807 19:47:45.621369 (+ 1984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:45.621375 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:45.621460 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:45.621549 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:45.623056 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.623727 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.625245 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.625659 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.625788 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:45.629204 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:45.629212 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:45.629217 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:45.629258 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:45.631977 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:45.632701 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:45.632705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:45.632708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:45.632839 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:45.633252 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:45.633257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:45.633259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:45.633545 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:45.633677 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:45.634432 (+ 755us) server_negotiation.cc:299] Negotiation successful
0807 19:47:45.634659 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 19:47:45.636037 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.618357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34683 (local address 127.0.0.1:44120)
0807 19:47:45.619191 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.619243 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:45.621060 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:45.621814 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:45.621822 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:45.622342 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:45.622887 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:45.622904 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.623884 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.623888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:45.625066 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:45.625085 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.625900 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.625908 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:45.626119 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:45.626739 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:45.626761 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:45.628946 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:45.632122 (+ 3176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:45.632131 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:45.632149 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:45.632583 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:45.632948 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:45.632952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:45.632956 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:45.633129 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:45.633673 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:45.633681 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:45.634042 (+ 361us) client_negotiation.cc:769] Sending connection context
0807 19:47:45.634304 (+ 262us) client_negotiation.cc:240] Negotiation successful
0807 19:47:45.634861 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":323,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.208.60@KRBTEST.COM" created.
I20260807 19:47:45.637776 24508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:45 dist-test-slave-vq2q krb5kdc[24334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.61@KRBTEST.COM for kudu/127.30.208.61@KRBTEST.COM
I20260807 19:47:45.652520 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.638528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37991 (local address 127.0.0.1:54644)
0807 19:47:45.638715 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.638733 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:45.638825 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:45.639802 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:45.639807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:45.640207 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:45.640352 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:45.640360 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.641728 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.641733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:45.642632 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:45.642645 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.643665 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.643670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:45.643833 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:45.644402 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:45.644420 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:45.646841 (+ 2421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:45.650189 (+ 3348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:45.650192 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:45.650195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:45.650563 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:45.650975 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:45.650979 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:45.650981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:45.651119 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:45.651822 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:45.651826 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:45.651943 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 19:47:45.652052 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 19:47:45.652251 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 19:47:45.652878 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.638789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54644 (local address 127.30.208.61:37991)
0807 19:47:45.639415 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:45.639421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:45.639445 (+ 24us) server_negotiation.cc:407] Connection header received
0807 19:47:45.639487 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:45.639492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:45.639661 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:45.639789 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:45.640509 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.641601 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.642789 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.643520 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.643634 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:45.647088 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:45.647104 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:45.647122 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:45.647156 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:45.650059 (+ 2903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:45.650707 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:45.650717 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:45.650719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:45.650866 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:45.651221 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:45.651225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:45.651229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:45.651673 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:45.651881 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:45.652249 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 19:47:45.652512 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":356,"threads_started":1}
I20260807 19:47:45.655252 24508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:47:45.656378 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.655843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41160)
0807 19:47:45.656022 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.656179 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 19:47:45.656601 24452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
I20260807 19:47:45.658126 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.657758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41168)
0807 19:47:45.657905 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.658017 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 19:47:45.658399 24508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:34203: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.208.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.60.keytab.
Entry for principal kudu/127.30.208.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.60.keytab.
I20260807 19:47:45.674875 31552 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
/tmp/dist-test-taskXLhxOI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.60:34203
--webserver_interface=127.30.208.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.60.keytab
--principal=kudu/127.30.208.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.208.20:35295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.208.62:34683,127.30.208.61:37991,127.30.208.60:34203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 19:47:45.707705 24508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } attempt: 1
I20260807 19:47:45.708937 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.708479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41184)
0807 19:47:45.708668 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.708784 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 19:47:45.709911 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.709582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41186)
0807 19:47:45.709717 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.709806 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 19:47:45.710093 24508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:34203: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
I20260807 19:47:45.778307 24508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } attempt: 2
I20260807 19:47:45.779634 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.779073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41190)
0807 19:47:45.779299 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.779413 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 19:47:45.780746 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.780347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41204)
0807 19:47:45.780496 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.780584 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 19:47:45.781199 24508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:34203: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
I20260807 19:47:45.791396 24425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } attempt: 5
Aug 07 19:47:45 dist-test-slave-vq2q krb5kdc[24334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for kudu/127.30.208.61@KRBTEST.COM
I20260807 19:47:45.804545 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.792124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37991 (local address 127.0.0.1:54652)
0807 19:47:45.792307 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.792328 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:45.792455 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:45.792802 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:45.792807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:45.793239 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:45.793436 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:45.793447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.794430 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.794434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:45.795285 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:45.795299 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.796030 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.796036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:45.796235 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:45.796865 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:45.796885 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:45.799246 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:45.802153 (+ 2907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:45.802159 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:45.802163 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:45.802529 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:45.802890 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:45.802895 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:45.802899 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:45.803034 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:45.803545 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:45.803549 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:45.803708 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 19:47:45.803876 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 19:47:45.804193 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 19:47:45.804546 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.792239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54652 (local address 127.30.208.61:37991)
0807 19:47:45.792439 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:45.792444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:45.792466 (+ 22us) server_negotiation.cc:407] Connection header received
0807 19:47:45.792567 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:45.792573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:45.792654 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:45.792737 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:45.793583 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.794296 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.795431 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:45.795885 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:45.795993 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:45.799436 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:45.799442 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:45.799447 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:45.799482 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:45.802020 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:45.802645 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:45.802649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:45.802652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:45.802779 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:45.803138 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:45.803141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:45.803143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:45.803430 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:45.803560 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:45.803991 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 19:47:45.804212 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 19:47:45.805728 24425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } has no permanent_uuid. Determining permanent_uuid...
I20260807 19:47:45.806901 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.806415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41206)
0807 19:47:45.806609 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.806712 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 19:47:45.807940 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.807550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41212)
0807 19:47:45.807695 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.807794 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 19:47:45.808145 24425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:34203: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
I20260807 19:47:45.859377 24425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } attempt: 1
I20260807 19:47:45.860687 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.860127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41226)
0807 19:47:45.860354 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.860471 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 19:47:45.861784 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.861368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41238)
0807 19:47:45.861537 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.861637 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 19:47:45.862013 24425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:34203: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
I20260807 19:47:45.872432 24508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } attempt: 3
I20260807 19:47:45.873600 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.873107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41248)
0807 19:47:45.873362 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.873450 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260807 19:47:45.874683 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.874262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41262)
0807 19:47:45.874487 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.874567 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260807 19:47:45.874912 24508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:34203: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
I20260807 19:47:45.930250 24425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } attempt: 2
I20260807 19:47:45.931603 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.931024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41266)
0807 19:47:45.931275 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.931395 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 19:47:45.932622 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:45.932251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41272)
0807 19:47:45.932425 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:45.932497 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260807 19:47:45.932893 24425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:34203: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
I20260807 19:47:46.018129 24508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } attempt: 4
I20260807 19:47:46.019364 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.018863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41274)
0807 19:47:46.019106 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:46.019215 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 19:47:46.020378 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.020055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41290)
0807 19:47:46.020184 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:46.020269 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 19:47:46.020628 24508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:34203: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
W20260807 19:47:46.022089 24515 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 19:47:46.022419 24515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 19:47:46.022512 24515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 19:47:46.022621 24515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 19:47:46.030251 24515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 19:47:46.030356 24515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 19:47:46.030448 24515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 19:47:46.030514 24515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 19:47:46.030597 24515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 19:47:46.039119 24425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } attempt: 3
I20260807 19:47:46.040256 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.039793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41304)
0807 19:47:46.040000 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:46.040090 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 19:47:46.040970 24515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.208.20:35295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.208.62:34683,127.30.208.61:37991,127.30.208.60:34203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/krb5kdc/kudu_127.30.208.60.keytab
--principal=kudu/127.30.208.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.208.60:34203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.208.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 18:41:41 UTC on ccycloud.quasar-unxwht.root.comops.site
build id 4573426
ASAN enabled
I20260807 19:47:46.041308 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.040875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41312)
0807 19:47:46.041079 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:46.041151 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260807 19:47:46.041575 24425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.208.60:34203: Network error: Client connection negotiation failed: client connection to 127.30.208.60:34203: connect: Connection refused (error 111)
I20260807 19:47:46.042371 24515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 19:47:46.043630 24515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 19:47:46.055519 24520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:46.055486 24522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 19:47:46.055473 24524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 19:47:46.056420 24515 server_base.cc:1034] running on GCE node
I20260807 19:47:46.057156 24515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 19:47:46.058638 24515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 19:47:46.059885 24515 hybrid_clock.cc:648] HybridClock initialized: now 1786132066059791 us; error 44 us; skew 500 ppm
I20260807 19:47:46.060202 24515 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 19:47:46 dist-test-slave-vq2q krb5kdc[24334](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 19:47:46.063256 24515 init.cc:377] Logged in from keytab as kudu/127.30.208.60@KRBTEST.COM (short username kudu)
I20260807 19:47:46.064627 24515 webserver.cc:460] Webserver started at http://127.30.208.60:39585/ using document root <none> and password file <none>
I20260807 19:47:46.065336 24515 fs_manager.cc:359] Metadata directory not provided
I20260807 19:47:46.065442 24515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 19:47:46.065763 24515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 19:47:46.068686 24515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/data/instance:
uuid: "d38bb9a07f7e4ce4a0796e71e869cf19"
format_stamp: "Formatted at 2026-08-07 19:47:46 on dist-test-slave-vq2q"
server_key: "46d2dd707df8cd90d93f6f115beb4f7d"
server_key_iv: "4ac0270dc9537afb841616221dc49c21"
server_key_version: "encryptionkey@0"
I20260807 19:47:46.069502 24515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/wal/instance:
uuid: "d38bb9a07f7e4ce4a0796e71e869cf19"
format_stamp: "Formatted at 2026-08-07 19:47:46 on dist-test-slave-vq2q"
server_key: "46d2dd707df8cd90d93f6f115beb4f7d"
server_key_iv: "4ac0270dc9537afb841616221dc49c21"
server_key_version: "encryptionkey@0"
I20260807 19:47:46.074450 24515 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260807 19:47:46.077950 24531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 19:47:46.079326 24515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 19:47:46.079497 24515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/data,/tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/wal
uuid: "d38bb9a07f7e4ce4a0796e71e869cf19"
format_stamp: "Formatted at 2026-08-07 19:47:46 on dist-test-slave-vq2q"
server_key: "46d2dd707df8cd90d93f6f115beb4f7d"
server_key_iv: "4ac0270dc9537afb841616221dc49c21"
server_key_version: "encryptionkey@0"
I20260807 19:47:46.079711 24515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 19:47:46.099852 24515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 19:47:46.103850 24515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 19:47:46.104104 24515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 19:47:46.121978 24515 rpc_server.cc:307] RPC server started. Bound to: 127.30.208.60:34203
I20260807 19:47:46.121999 24591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.208.60:34203 every 8 connection(s)
I20260807 19:47:46.123354 24515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/data/info.pb
I20260807 19:47:46.125967 31552 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu as pid 24515
I20260807 19:47:46.126134 31552 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXLhxOI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786131702042267-31552-0/minicluster-data/master-2/wal/instance
I20260807 19:47:46.126569 31552 external_mini_cluster.cc:1428] Setting key 6cf8f75a57d2e7baf315453b71c16557
I20260807 19:47:46.127213 24592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 19:47:46.137239 24592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:46 dist-test-slave-vq2q krb5kdc[24334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.60@KRBTEST.COM
Aug 07 19:47:46 dist-test-slave-vq2q krb5kdc[24334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.60@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
I20260807 19:47:46.150315 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.128336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41326 (local address 127.30.208.60:34203)
0807 19:47:46.129167 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.129210 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.129262 (+ 52us) server_negotiation.cc:407] Connection header received
0807 19:47:46.130896 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.130924 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.131448 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.131907 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.133298 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.135303 (+ 2005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.136689 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.137767 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.137976 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.142545 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:46.142553 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:46.142563 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:46.142604 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:46.145610 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.146332 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.146339 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.146344 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.146545 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.146929 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.146956 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.146958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.147483 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:46.147750 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:46.148749 (+ 999us) server_negotiation.cc:299] Negotiation successful
0807 19:47:46.149206 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":331,"threads_started":1}
I20260807 19:47:46.154536 24597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.140067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.62:34683 (local address 127.0.0.1:44128)
0807 19:47:46.140872 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:46.140934 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:46.141111 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:46.141847 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:46.141855 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:46.142470 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:46.142797 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:46.142807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.143892 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.143896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:46.144991 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:46.145003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.145860 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.145866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:46.146019 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.146556 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:46.146577 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:46.148847 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:46.151878 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:46.151890 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:46.151897 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:46.152306 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:46.152693 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:46.152696 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:46.152699 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:46.152831 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:46.153496 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:46.153504 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:46.153680 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 19:47:46.153839 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 19:47:46.154157 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":326,"threads_started":1}
I20260807 19:47:46.154682 24598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.140125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44128 (local address 127.30.208.62:34683)
0807 19:47:46.141494 (+ 1369us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.141501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.141522 (+ 21us) server_negotiation.cc:407] Connection header received
0807 19:47:46.141581 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.141584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.141655 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.141755 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.142960 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.143729 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.145135 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.145662 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.145786 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.149077 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:46.149084 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:46.149090 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:46.149128 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:46.151748 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.152421 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.152427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.152430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.152568 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.152938 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.152942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.152944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.153331 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:46.153499 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:46.154044 (+ 545us) server_negotiation.cc:299] Negotiation successful
0807 19:47:46.154288 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1242,"thread_start_us":1147,"threads_started":1}
I20260807 19:47:46.156052 24592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:46 dist-test-slave-vq2q krb5kdc[24334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.62@KRBTEST.COM
Aug 07 19:47:46 dist-test-slave-vq2q krb5kdc[24334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.60@KRBTEST.COM for kudu/127.30.208.61@KRBTEST.COM
I20260807 19:47:46.167971 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.153329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44138 (local address 127.30.208.62:34683)
0807 19:47:46.154024 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.154028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.154047 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:47:46.154084 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.154088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.154138 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.154237 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.155674 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.156497 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.158097 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.158572 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.158695 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.162741 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:46.162749 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:46.162754 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:46.162786 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:46.165402 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.166070 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.166074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.166076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.166195 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.166555 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.166559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.166561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.166812 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:46.167073 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:46.167380 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 19:47:46.167594 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":345,"threads_started":1}
I20260807 19:47:46.168388 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.156773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54654 (local address 127.30.208.61:37991)
0807 19:47:46.156957 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.156962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.156981 (+ 19us) server_negotiation.cc:407] Connection header received
0807 19:47:46.157089 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.157094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.157171 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.157313 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.157979 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.158698 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.159964 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.160371 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.160475 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.163322 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:46.163327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:46.163331 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:46.163361 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:46.165973 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.166577 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.166581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.166583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.166713 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.167060 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.167064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.167066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.167328 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:46.167460 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:46.167843 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 19:47:46.168038 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:47:46.168728 24597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.156748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.61:37991 (local address 127.0.0.1:54654)
0807 19:47:46.156907 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:46.156925 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:46.157005 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:46.157348 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:46.157351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:46.157682 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:46.157823 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:46.157832 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.158829 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.158835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:46.159826 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:46.159839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.160509 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.160514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:46.160682 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.161140 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:46.161158 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:46.163150 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:46.166097 (+ 2947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:46.166103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:46.166107 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:46.166468 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:46.166815 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:46.166820 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:46.166824 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:46.166962 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:46.167444 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:46.167448 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:46.167587 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 19:47:46.167737 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 19:47:46.168012 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 19:47:46.169848 24592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 19:47:46 dist-test-slave-vq2q krb5kdc[24334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.60@KRBTEST.COM for kudu/127.30.208.60@KRBTEST.COM
Aug 07 19:47:46 dist-test-slave-vq2q krb5kdc[24334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132064, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.208.61@KRBTEST.COM
I20260807 19:47:46.182868 24597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.170493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41336)
0807 19:47:46.170636 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:46.170651 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:46.170721 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:46.171119 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:46.171123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:46.171507 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:46.171675 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:46.171686 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.172613 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.172617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:46.173527 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:46.173539 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.174274 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.174280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:46.174431 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.175058 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:46.175074 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:46.177229 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:46.180433 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:46.180437 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:46.180439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:46.180815 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:46.181257 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:46.181261 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:46.181263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:46.181391 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:46.181918 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:46.181922 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:46.182050 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 19:47:46.182177 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 19:47:46.182484 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 19:47:46.183040 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.170771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41336 (local address 127.30.208.60:34203)
0807 19:47:46.170907 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.170912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.170924 (+ 12us) server_negotiation.cc:407] Connection header received
0807 19:47:46.170956 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.170959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.171009 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.171085 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.171816 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.172490 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.173655 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.174112 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.174224 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.177394 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:46.177400 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:46.177405 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:46.177436 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:46.180282 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.180933 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.180939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.180942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.181087 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.181494 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.181498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.181501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.181808 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:46.181934 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:46.182292 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 19:47:46.182655 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":135,"server-negotiator.queue_time_us":52}
I20260807 19:47:46.184263 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.169730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54664 (local address 127.30.208.61:37991)
0807 19:47:46.169899 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.169904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.170011 (+ 107us) server_negotiation.cc:407] Connection header received
0807 19:47:46.170133 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.170137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.170209 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.170324 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.171602 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.172405 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.173694 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.174097 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.174194 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.178466 (+ 4272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:46.178473 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:46.178478 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:46.178516 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:46.181467 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.182106 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.182112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.182116 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.182256 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.182598 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.182602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.182604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.182904 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:46.183188 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:46.183693 (+ 505us) server_negotiation.cc:299] Negotiation successful
0807 19:47:46.183961 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 19:47:46.188933 24592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19: Bootstrap starting.
I20260807 19:47:46.189379 31552 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 19:47:46.192874 24592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:46.194226 24592 log.cc:824] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:46.192404 24397 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9c13d3955af84691b14a3bb3ce24a891"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44138
I20260807 19:47:46.197445 24592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19: No bootstrap required, opened a new log
I20260807 19:47:46.208323 24592 raft_consensus.cc:348] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } }
I20260807 19:47:46.208758 24592 raft_consensus.cc:729] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d38bb9a07f7e4ce4a0796e71e869cf19, State: Initialized, Role: FOLLOWER
I20260807 19:47:46.209280 24592 consensus_queue.cc:260] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } }
I20260807 19:47:46.210318 24602 sys_catalog.cc:455] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } } }
I20260807 19:47:46.210695 24602 sys_catalog.cc:458] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:46.211525 24592 sys_catalog.cc:564] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:46.218794 24425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } attempt: 4
W20260807 19:47:46.221305 24615 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 19:47:46.221468 24615 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 07 19:47:46 dist-test-slave-vq2q krb5kdc[24334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.62@KRBTEST.COM for kudu/127.30.208.60@KRBTEST.COM
I20260807 19:47:46.219044 24515 master_runner.cc:186] Error getting master registration for 127.30.208.62:34683: OK, instance_id { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" instance_seqno: 1786132065079603 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:47:46.231330 24515 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:47:46.236997 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.220429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41346 (local address 127.30.208.60:34203)
0807 19:47:46.220655 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.220661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.220689 (+ 28us) server_negotiation.cc:407] Connection header received
0807 19:47:46.220748 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.220753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.220832 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.220937 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.222197 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.222999 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.224793 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.225492 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.225630 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.229898 (+ 4268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:46.229905 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:46.229910 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:46.229960 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:46.233912 (+ 3952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.234786 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.234792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.234796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.234945 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.235380 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.235385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.235389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.235776 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:46.235948 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:46.236403 (+ 455us) server_negotiation.cc:299] Negotiation successful
0807 19:47:46.236631 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 19:47:46.237005 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.219501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41346)
0807 19:47:46.219855 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:46.219875 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:46.219988 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:46.221164 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:46.221290 (+ 126us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:46.221810 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:46.222037 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:46.222051 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.223138 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.223144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:46.224654 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:46.224669 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.225649 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.225655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:46.225862 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.226532 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:46.226557 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:46.229391 (+ 2834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:46.234070 (+ 4679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:46.234076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:46.234080 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:46.234663 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:46.235060 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:46.235066 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:46.235070 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:46.235252 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:46.235920 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:46.235926 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:46.236117 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 19:47:46.236279 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 19:47:46.236606 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227}
I20260807 19:47:46.243364 24425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891: Bootstrap starting.
I20260807 19:47:46.247896 24425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:46.249361 24425 log.cc:824] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:46.252929 24425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891: No bootstrap required, opened a new log
I20260807 19:47:46.260767 24425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } }
I20260807 19:47:46.261237 24425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c13d3955af84691b14a3bb3ce24a891, State: Initialized, Role: FOLLOWER
I20260807 19:47:46.261808 24425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } }
I20260807 19:47:46.262832 24619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } } }
I20260807 19:47:46.263247 24619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:46.264242 24425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:46.264309 24397 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 19:47:46.264515 24397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:46.267302 24397 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } }
I20260807 19:47:46.270540 24397 leader_election.cc:288] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [CANDIDATE]: Term 1 election: Requested vote from peers 411480b61aa14fc8b56ab4819c79bf85 (127.30.208.61:37991), d38bb9a07f7e4ce4a0796e71e869cf19 (127.30.208.60:34203)
I20260807 19:47:46.270641 24563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c13d3955af84691b14a3bb3ce24a891" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19"
I20260807 19:47:46.271176 24563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:46.274979 24563 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c13d3955af84691b14a3bb3ce24a891 in term 1.
I20260807 19:47:46.275903 24369 leader_election.cc:302] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c13d3955af84691b14a3bb3ce24a891, d38bb9a07f7e4ce4a0796e71e869cf19; no voters:
I20260807 19:47:46.276439 24619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 19:47:46.287720 24619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [term 1 LEADER]: Becoming Leader. State: Replica: 9c13d3955af84691b14a3bb3ce24a891, State: Running, Role: LEADER
I20260807 19:47:46.288517 24619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } }
I20260807 19:47:46.310523 24626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c13d3955af84691b14a3bb3ce24a891. Latest consensus state: current_term: 1 leader_uuid: "9c13d3955af84691b14a3bb3ce24a891" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } } }
I20260807 19:47:46.311093 24626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:46.313441 24637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260807 19:47:46.319206 24374 tablet.cc:2369] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 19:47:46.319828 24508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } attempt: 5
I20260807 19:47:46.328826 24637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 19:47:46.341603 24642 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 19:47:46.342891 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.300620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44148 (local address 127.30.208.62:34683)
0807 19:47:46.300842 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.300848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.313652 (+ 12804us) server_negotiation.cc:407] Connection header received
0807 19:47:46.315284 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.315291 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.315403 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.315498 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.317747 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.318727 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.327022 (+ 8295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.327766 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.327877 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.333002 (+ 5125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:46.333010 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:46.333015 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:46.333062 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:46.336749 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.340224 (+ 3475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.340229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.340232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.340424 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.340903 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.340908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.340911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.341313 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:46.341568 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:46.342146 (+ 578us) server_negotiation.cc:299] Negotiation successful
0807 19:47:46.342439 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 19:47:46.351243 24596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.314172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41348 (local address 127.30.208.60:34203)
0807 19:47:46.314393 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.314403 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.318377 (+ 3974us) server_negotiation.cc:407] Connection header received
0807 19:47:46.318603 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.318609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.318682 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.318764 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.323733 (+ 4969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.324960 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.331458 (+ 6498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.332289 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.332403 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.344004 (+ 11601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:46.344011 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:46.344015 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:46.344061 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:46.347860 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.348767 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.348772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.348776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.348950 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.349433 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.349439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.349442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.349808 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:46.350062 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:46.350618 (+ 556us) server_negotiation.cc:299] Negotiation successful
0807 19:47:46.350870 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 19:47:46.356287 24538 tablet.cc:2369] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 19:47:46 dist-test-slave-vq2q krb5kdc[24334](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786132065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.208.61@KRBTEST.COM for kudu/127.30.208.60@KRBTEST.COM
I20260807 19:47:46.365509 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.309724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54672 (local address 127.30.208.61:37991)
0807 19:47:46.309911 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.309917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.316425 (+ 6508us) server_negotiation.cc:407] Connection header received
0807 19:47:46.316590 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.316596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.316682 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.316788 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.330166 (+ 13378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.331303 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.335065 (+ 3762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.335713 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.335813 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.358650 (+ 22837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:46.358659 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:46.358664 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:46.358707 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:46.362214 (+ 3507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.363071 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.363078 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.363081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.363247 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.363714 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.363719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.363722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.364107 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:46.364338 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:46.364916 (+ 578us) server_negotiation.cc:299] Negotiation successful
0807 19:47:46.365149 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 19:47:46.369061 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.320371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.208.60:34203 (local address 127.0.0.1:41362)
0807 19:47:46.320519 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 19:47:46.320538 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 19:47:46.320635 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 19:47:46.336679 (+ 16044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 19:47:46.336684 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 19:47:46.337153 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0807 19:47:46.337394 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:46.337408 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.340386 (+ 2978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.340391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:46.341772 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 19:47:46.341788 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.353530 (+ 11742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.353549 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 19:47:46.353833 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.354538 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 19:47:46.354562 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 19:47:46.358752 (+ 4190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 19:47:46.365936 (+ 7184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:46.365941 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:46.365957 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:46.366449 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:46.367117 (+ 668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 19:47:46.367122 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 19:47:46.367125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 19:47:46.367310 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 19:47:46.368114 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 19:47:46.368119 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 19:47:46.368253 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 19:47:46.368368 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 19:47:46.368667 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 19:47:46.369515 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:46.320720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41362 (local address 127.30.208.60:34203)
0807 19:47:46.336110 (+ 15390us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.336116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.336142 (+ 26us) server_negotiation.cc:407] Connection header received
0807 19:47:46.336258 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.336263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.336401 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.337604 (+ 1203us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.338120 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.340169 (+ 2049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.342140 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:46.353131 (+ 10991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:46.354218 (+ 1087us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:46.359122 (+ 4904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 19:47:46.359130 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 19:47:46.359135 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 19:47:46.359199 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 19:47:46.365371 (+ 6172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.366688 (+ 1317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.366695 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.366698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.366953 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 19:47:46.367494 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 19:47:46.367498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 19:47:46.367501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 19:47:46.367990 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 19:47:46.368116 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:46.368508 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 19:47:46.368716 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15137,"thread_start_us":14685,"threads_started":1}
I20260807 19:47:46.347481 24351 master_runner.cc:186] Error getting master registration for 127.30.208.61:37991: OK, instance_id { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" instance_seqno: 1786132065595145 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 19:47:46.373847 24351 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 19:47:46.376924 24563 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19 [term 1 FOLLOWER]: Refusing update from remote peer 9c13d3955af84691b14a3bb3ce24a891: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:46.377753 24508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85: Bootstrap starting.
I20260807 19:47:46.379719 24626 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:46.390106 24508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85: Neither blocks nor log segments found. Creating new log.
I20260807 19:47:46.392748 24508 log.cc:824] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85: Log is configured to *not* fsync() on all Append() calls
I20260807 19:47:46.405596 24508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85: No bootstrap required, opened a new log
I20260807 19:47:46.423862 24602 sys_catalog.cc:455] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c13d3955af84691b14a3bb3ce24a891. Latest consensus state: current_term: 1 leader_uuid: "9c13d3955af84691b14a3bb3ce24a891" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } } }
I20260807 19:47:46.424189 24602 sys_catalog.cc:458] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:46.427870 24626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c13d3955af84691b14a3bb3ce24a891" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } } }
I20260807 19:47:46.428159 24626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:46.437493 24637 catalog_manager.cc:1347] Generated new cluster ID: 67df560cebf8448e8a577a11bb2a850a
I20260807 19:47:46.437618 24637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 19:47:46.437592 24508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } }
I20260807 19:47:46.438277 24508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 411480b61aa14fc8b56ab4819c79bf85, State: Initialized, Role: FOLLOWER
I20260807 19:47:46.439079 24508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } }
I20260807 19:47:46.441016 24602 sys_catalog.cc:455] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c13d3955af84691b14a3bb3ce24a891" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } } }
I20260807 19:47:46.441813 24602 sys_catalog.cc:458] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:46.452353 24481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c13d3955af84691b14a3bb3ce24a891" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "411480b61aa14fc8b56ab4819c79bf85"
I20260807 19:47:46.452694 24480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85 [term 0 FOLLOWER]: Advancing to term 1
I20260807 19:47:46.452359 24508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 19:47:46.456555 24480 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85 [term 1 FOLLOWER]: Refusing update from remote peer 9c13d3955af84691b14a3bb3ce24a891: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 19:47:46.457917 24637 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 19:47:46.459801 24637 catalog_manager.cc:1504] Loading token signing keys...
I20260807 19:47:46.459923 24481 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c13d3955af84691b14a3bb3ce24a891 in term 1.
I20260807 19:47:46.462991 24626 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [LEADER]: Connected to new peer: Peer: permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 19:47:46.467681 24651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } } }
I20260807 19:47:46.468089 24651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:46.476881 24637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891: Generated new TSK 0
I20260807 19:47:46.477980 24637 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 19:47:46.525056 24651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c13d3955af84691b14a3bb3ce24a891. Latest consensus state: current_term: 1 leader_uuid: "9c13d3955af84691b14a3bb3ce24a891" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } } }
I20260807 19:47:46.525468 24651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:46.525826 24654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c13d3955af84691b14a3bb3ce24a891" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } } }
I20260807 19:47:46.526154 24654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 19:47:46.527865 24619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c13d3955af84691b14a3bb3ce24a891" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9c13d3955af84691b14a3bb3ce24a891" member_type: VOTER last_known_addr { host: "127.30.208.62" port: 34683 } } peers { permanent_uuid: "411480b61aa14fc8b56ab4819c79bf85" member_type: VOTER last_known_addr { host: "127.30.208.61" port: 37991 } } peers { permanent_uuid: "d38bb9a07f7e4ce4a0796e71e869cf19" member_type: VOTER last_known_addr { host: "127.30.208.60" port: 34203 } } }
I20260807 19:47:46.528239 24619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c13d3955af84691b14a3bb3ce24a891 [sys.catalog]: This master's current role is: LEADER
I20260807 19:47:46.589835 24670 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260807 19:47:46.600407 31552 mini_kdc.cc:348] Time spent kdestroy: real 0.102s user 0.000s sys 0.033s
W20260807 19:47:46.644744 24599 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:46.631959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44164 (local address 127.30.208.62:34683)
0807 19:47:46.632162 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.632168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.642049 (+ 9881us) server_negotiation.cc:407] Connection header received
0807 19:47:46.642759 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.642808 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.642947 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.643036 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.644400 (+ 1364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44164: BlockingRecv error: recv got EOF from 127.0.0.1:44164 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 19:47:46.646764 24512 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:46.637792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54682 (local address 127.30.208.61:37991)
0807 19:47:46.638009 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.638017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.644298 (+ 6281us) server_negotiation.cc:407] Connection header received
0807 19:47:46.645216 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.645222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.645317 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.645385 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.646531 (+ 1146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54682: BlockingRecv error: recv got EOF from 127.0.0.1:54682 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 19:47:46.647336 24641 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:46.641572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41364 (local address 127.30.208.60:34203)
0807 19:47:46.641753 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.641760 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.645583 (+ 3823us) server_negotiation.cc:407] Connection header received
0807 19:47:46.645923 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.645929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.646030 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.646119 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.647095 (+ 976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41364: BlockingRecv error: recv got EOF from 127.0.0.1:41364 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 19:47:46.655695 24512 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:46.653325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54692 (local address 127.30.208.61:37991)
0807 19:47:46.653524 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.653531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.654152 (+ 621us) server_negotiation.cc:407] Connection header received
0807 19:47:46.654366 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.654372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.654535 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.654643 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.655440 (+ 797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54692: BlockingRecv error: recv got EOF from 127.0.0.1:54692 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 19:47:46.665380 24512 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:46.663250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54702 (local address 127.30.208.61:37991)
0807 19:47:46.663405 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.663411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.663896 (+ 485us) server_negotiation.cc:407] Connection header received
0807 19:47:46.664058 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.664064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.664158 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.664323 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.665099 (+ 776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54702: BlockingRecv error: recv got EOF from 127.0.0.1:54702 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 19:47:46.675158 24512 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:46.672322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54708 (local address 127.30.208.61:37991)
0807 19:47:46.672498 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.672504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.673713 (+ 1209us) server_negotiation.cc:407] Connection header received
0807 19:47:46.673912 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.673918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.674026 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.674143 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.674911 (+ 768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54708: BlockingRecv error: recv got EOF from 127.0.0.1:54708 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 19:47:46.685809 24512 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:46.683086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54712 (local address 127.30.208.61:37991)
0807 19:47:46.683311 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.683317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.683890 (+ 573us) server_negotiation.cc:407] Connection header received
0807 19:47:46.684169 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.684176 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.684286 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.684397 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.685547 (+ 1150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54712: BlockingRecv error: recv got EOF from 127.0.0.1:54712 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
W20260807 19:47:46.700374 24512 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:46.698102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54726 (local address 127.30.208.61:37991)
0807 19:47:46.698323 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.698330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.698901 (+ 571us) server_negotiation.cc:407] Connection header received
0807 19:47:46.699104 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.699110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.699209 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.699323 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.700132 (+ 809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54726: BlockingRecv error: recv got EOF from 127.0.0.1:54726 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 19:47:46.722479 24512 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:46.720132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54730 (local address 127.30.208.61:37991)
0807 19:47:46.720359 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.720366 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.720894 (+ 528us) server_negotiation.cc:407] Connection header received
0807 19:47:46.721119 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.721125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.721253 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.721352 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.722237 (+ 885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54730: BlockingRecv error: recv got EOF from 127.0.0.1:54730 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 19:47:46.760625 24512 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:46.758151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54736 (local address 127.30.208.61:37991)
0807 19:47:46.758367 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.758374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.758960 (+ 586us) server_negotiation.cc:407] Connection header received
0807 19:47:46.759272 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.759279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.759381 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.759491 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.760358 (+ 867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54736: BlockingRecv error: recv got EOF from 127.0.0.1:54736 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260807 19:47:46.829723 24512 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:46.827900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54752 (local address 127.30.208.61:37991)
0807 19:47:46.828128 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.828135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.828370 (+ 235us) server_negotiation.cc:407] Connection header received
0807 19:47:46.828536 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.828541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.828630 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.828755 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.829499 (+ 744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54752: BlockingRecv error: recv got EOF from 127.0.0.1:54752 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 19:47:46.963188 24512 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:46.961365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54768 (local address 127.30.208.61:37991)
0807 19:47:46.961539 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:46.961543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:46.961855 (+ 312us) server_negotiation.cc:407] Connection header received
0807 19:47:46.962045 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:46.962050 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:46.962125 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:46.962204 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:46.962929 (+ 725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54768: BlockingRecv error: recv got EOF from 127.0.0.1:54768 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:47:47.224170 24615 catalog_manager.cc:1259] Loaded cluster ID: 67df560cebf8448e8a577a11bb2a850a
I20260807 19:47:47.224311 24615 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19: loading cluster ID for follower catalog manager: success
I20260807 19:47:47.227778 24615 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19: acquiring CA information for follower catalog manager: success
W20260807 19:47:47.227972 24512 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 19:47:47.226117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54772 (local address 127.30.208.61:37991)
0807 19:47:47.226327 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:47.226334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:47.226562 (+ 228us) server_negotiation.cc:407] Connection header received
0807 19:47:47.226795 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:47.226799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:47.226873 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:47.226964 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 19:47:47.227753 (+ 789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54772: BlockingRecv error: recv got EOF from 127.0.0.1:54772 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 19:47:47.230108 24615 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d38bb9a07f7e4ce4a0796e71e869cf19: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:47:47.594149 24670 catalog_manager.cc:1259] Loaded cluster ID: 67df560cebf8448e8a577a11bb2a850a
I20260807 19:47:47.594295 24670 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85: loading cluster ID for follower catalog manager: success
I20260807 19:47:47.596953 24670 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85: acquiring CA information for follower catalog manager: success
I20260807 19:47:47.598881 24670 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 411480b61aa14fc8b56ab4819c79bf85: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 19:47:47.751783 24748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:47.744563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.30.208.61:37991)
0807 19:47:47.745283 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:47.745289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:47.745482 (+ 193us) server_negotiation.cc:407] Connection header received
0807 19:47:47.745728 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:47.745733 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:47.745897 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:47.746015 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:47.746487 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:47.747418 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:47.749394 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:47.749927 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:47.750018 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:47.750568 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:47.751157 (+ 589us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:47.751278 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:47.751406 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 19:47:47.751480 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":276,"threads_started":1}
I20260807 19:47:47.766419 24754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:47.760258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41380 (local address 127.30.208.60:34203)
0807 19:47:47.761143 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:47.761148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:47.761163 (+ 15us) server_negotiation.cc:407] Connection header received
0807 19:47:47.761332 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:47.761338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:47.761497 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:47.761646 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:47.762065 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:47.762758 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:47.764091 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:47.764508 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:47.764619 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:47.765203 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:47.765783 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:47.765917 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:47.766029 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 19:47:47.766114 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":685,"threads_started":1}
I20260807 19:47:47.784649 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:47.772406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44166 (local address 127.30.208.62:34683)
0807 19:47:47.775944 (+ 3538us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:47.775950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:47.777109 (+ 1159us) server_negotiation.cc:407] Connection header received
0807 19:47:47.777258 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:47.777263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:47.777488 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:47.777658 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:47.778411 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:47.779519 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:47.781556 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:47.782208 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:47.782381 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:47.783140 (+ 759us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:47.783702 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:47.783814 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:47.784262 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 19:47:47.784349 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3413,"thread_start_us":6149,"threads_started":1}
I20260807 19:47:47.786863 24748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:47.772426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.30.208.61:37991)
0807 19:47:47.772589 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:47.772595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:47.779181 (+ 6586us) server_negotiation.cc:407] Connection header received
0807 19:47:47.779472 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:47.779477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:47.779565 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:47.779636 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:47.780510 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:47.781697 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:47.784001 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:47.784597 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:47.784709 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:47.786070 (+ 1361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:47.786199 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:47.786270 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:47.786504 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 19:47:47.786586 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 19:47:47.790256 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 24351
I20260807 19:47:47.791276 24419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:47.791440 24754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 19:47:47.784552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41392 (local address 127.30.208.60:34203)
0807 19:47:47.784730 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 19:47:47.784736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 19:47:47.785333 (+ 597us) server_negotiation.cc:407] Connection header received
0807 19:47:47.785565 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 19:47:47.785570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 19:47:47.785658 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 19:47:47.785773 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 19:47:47.786186 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:47.787267 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:47.789254 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 19:47:47.789905 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 19:47:47.790010 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 19:47:47.790616 (+ 606us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 19:47:47.790837 (+ 221us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 19:47:47.790917 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 19:47:47.791076 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 19:47:47.791171 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 19:47:47.905571 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 24351
I20260807 19:47:47.936455 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 24432
I20260807 19:47:47.937588 24503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:48.052872 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 24432
I20260807 19:47:48.072652 31552 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu pid 24515
I20260807 19:47:48.073764 24585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 19:47:48.181357 31552 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXLhxOI/build/asan/bin/kudu with pid 24515
2026-08-07T19:47:48Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4978 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9391 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (366103 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST